Kurate.org
Rankings
Back to Rankings

EvoMemState: Evolving Memory State for Personal Agents

Zhishang Xiang, Yunbo Tang, Zhongqiao Liu, Junkai Chen, Zerui Chen, Qingguo Hu, Yuxuan Zhang, William Grant

Nov 2, 2026arXiv:2611.03743v1
cs.IRcs.AI
Share
Scorecard· 16/16
6.0/10 impact

Coherent, well-evaluated framework with a reusable RL-trained memory-use advisor, but gains are concentrated on closely aligned benchmarks and mixed on standard recall

Abstract

Long-term memory enables large language model (LLM) agents to provide continuous personalized assistance by retaining user information across interactions. Existing systems support this process through fact extraction, organization, updating, and retrieval, but largely maintain memories as independent facts. This fact-centric paradigm may overlook how new facts update, constrain, or conflict with earlier ones, while leaving their applicability, role, and priority for the current request implicit. In this paper, we introduce EvoMemState, a state-centric framework that augments persistent facts with two complementary states: Relational States that capture how facts remain valid and relate to one another as memory evolves, and a Reasoning State that determines how these facts should be applied to the current request. Specifically, EvoMemState consists of three key components: (i) Ontology-Guided Memory State Induction extracts Persistent Facts and constructs Relational States that maintain their validity, scope, and evolving relations. (ii) RL-Based State Evolution internalizes memory-use experience into a policy through reinforcement learning to generate a request-specific Reasoning State. (iii) State-Aware Reasoning then combines the relevant facts with their Relational States and the generated Reasoning State to guide the personal agent. Extensive experiments on long-term dialogue and memory-use benchmarks demonstrate consistent improvements over existing memory systems in retrieval completeness and memory-grounded response accuracy. Our code and implementation details are available at https://github.com/XMUDeepLIT/EvoMemState.git.

AI Impact Assessments

(1 model)

Core Contribution

EvoMemState argues that long-term memory for LLM personal agents should be "state-centric" rather than "fact-centric." It has three parts:

  • Ontology-guided induction: Persistent Facts are extracted incrementally into a concept/fact graph with typed SUPERSEDES and CONFLICTS_WITH edges. This yields Relational States that record validity, scope, and relations without overwriting history.
  • RL-trained Reasoning State generator: A small policy (Qwen3-4B) is trained with GRPO to emit a short natural-language "dynamic rule" for each request. The rule says which memories apply, in what role, and with what priority. The reward is the score gain of a frozen answer model with the rule versus without it.
  • Relation-traversal retrieval: K seed facts are expanded bidirectionally along the relation edges, so updates and conflicts come along with them.

    • The conceptual framing is a clear organizing idea. The system is a sensible combination of three known ingredients: temporal/supersession memory graphs (Zep, Supermemory, PGMem), query-time guidance generation, and RL-trained helper models (Memory-R1 and similar). The most distinctive element is training a decoupled, model-agnostic "memory-use advisor" with a counterfactual reward against a no-guidance baseline, while the answer model stays frozen.

    The preliminary studies are a useful motivation. They show that answer accuracy depends on complete retrieval, and that a fixed self-reflection prompt does not help even when retrieval is complete.

    Methodological Rigor

    *Strengths:*

  • The evaluation covers four benchmarks (MemSyco-Bench, MemTrapBench, LoCoMo, LoCoMo-Refined), two answer models, and nine baselines.
  • Baseline hyperparameters are shared across systems (bge-m3, top-k 10).
  • The ablation is informative. Removing the Reasoning State, the RL training, or the Relational State each costs roughly 4–7 points of overall score.
  • The authors separate retrieval completeness from evidence use through Acc@complete.
  • The cross-system transfer experiment (reasoning states added to six baselines) and the comparison against other memory-use strategies (Self-Recheck, MemGate, Dynamic Partition) are good controls.

    • *Weaknesses:*

  • Train/test proximity. The policy is trained on about 60K instances constructed from FEVER, SciFact, and PersonaMem. These were built to mimic the sycophancy and preference-update scenarios that MemSyco-Bench and MemTrapBench test. The authors of this paper also wrote MemSyco-Bench, and the benchmark is self-referential, so the large gains on those benchmarks may partly reflect distribution alignment rather than general memory-use ability. This is the central concern.
  • Uneven wins. On LoCoMo-Refined and LoCoMo the gains are small or inconsistent. SimpleMem beats EvoMemState on LoCoMo token F1 and on the Qwen3-8B LoCoMo average. Gains on MemTrapBench are modest. On Qwen3-8B, EvoMemState is below LightMem and SimpleMem on some MemTrapBench subtasks (Cognitive Bias, Trauma). The paper's "consistent improvements" framing is overstated.
  • Missing controls.

    • - There are no seeds, variances, or significance tests.

    - Several results are very close.

    - Baselines such as NaiveRAG, Mem0, and Supermemory score implausibly low on LoCoMo-Refined (Supermemory retrieval hit rate of 0.006), which suggests integration or configuration problems that favor the proposed method.

    - There is no ablation separating ontology-based traversal from the memory construction prompt, and no comparison of an RL-trained policy against a larger prompted model of equal size.

    - The Self-Guide comparison partly addresses this.

  • Preliminary-study framing. The self-reflection "drop" is only 1.6 points or less, within noise, yet it is used as evidence.
  • Cost. The LLM-heavy ontology construction (DeepSeek-V4-Flash) is not compared against baselines in cost, though the efficiency analysis does show shorter retrieved contexts.

    • Potential Impact

    The idea that memory use, not just retrieval, is a bottleneck is timely, and the plug-in reasoning-state policy is reusable. It improved all six baselines it was tested on, though by 0.2–4.9 points. The shared prompt and training recipe, along with code release, will help adoption. Practical applications include personalized assistants that need to handle preference changes, scope control, and conflicting evidence, and mitigations of memory-induced sycophancy. The reward design (gain over a no-guidance baseline) is a transferable recipe.

    The impact is likely confined to the LLM agent-memory community, a crowded, fast-moving area where many systems are published and few become standard. Reliance on very recent, same-group benchmarks limits how widely the results will be trusted. The approach also adds complexity (an LLM-built ontology and a separate policy model) versus simple baselines such as SimpleMem, which is strong on LoCoMo.

    Timeliness

    Very timely. Agent memory, memory sycophancy, and over-personalization are active topics, and RL for memory management is popular. The paper bundles several of these threads coherently.

    Strengths and Limitations

    *Strengths:*

  • A clear conceptual decomposition (facts, relational states, reasoning state).
  • A model-agnostic, frozen-answerer RL setup.
  • Thorough evaluation breadth and appendices (prompts, hyperparameters, configs, case studies).
  • Honest reporting of mixed results in the appendix, for example the Dynamic Rules drop on GPT-4o-mini LoCoMo-Refined, and that guidance alone does not help baselines.

    • *Limitations:*

  • The training data is aligned with the evaluation scenarios, and the benchmarks are partly self-authored.
  • No variance reporting.
  • Gains are concentrated on memory-use benchmarks and are smaller on standard recall.
  • The "state" terminology is somewhat inflated relative to what is implemented, which is a graph with edge types plus a short LLM-generated instruction.
  • The ontology depends on an LLM extractor whose error rates are not analyzed.
  • Only text is handled.
  • Generalization to truly out-of-distribution memory-use situations is untested.

    • Overall: A solid, well-engineered incremental-to-moderate contribution, with a useful decomposition and a reusable policy component. Its headline claims are tempered by benchmark/training alignment and uneven gains.

    Rating:6/ 10
    Significance 5.5Rigor 5.5Novelty 5.5Clarity 6.5

    Generated Nov 4, 2026

    Comparison History (0)

    No comparisons yet.