Who Schedules Agent Memory? A Sidecar Approach to Long-Horizon Tasks

Qijia Zhuang, Rui Liu, Yuxiang Ren

Oct 31, 2026arXiv:2611.00779v1

cs.AI

Scorecard· 16/16

6.5/10 impact

Timely, well-framed systems contribution with consistent gains in accuracy and latency on two benchmarks, but incremental in mechanism and lacking variance/diagnostics.

Abstract

As large language model agents increasingly undertake complex, long-horizon tasks, memory has become an important component for supporting sustained decision-making and action. Existing methods primarily focus on memory storage, updating, and retrieval, but often overlook how memory scheduling affects task execution: the main agent must identify memory needs, initiate retrieval, and wait for results while advancing the task, increasing both scheduling burden and execution time. To address this problem, we propose MemSidecar, which decouples memory scheduling from task execution through two independently selectable modes. Hooks directly delivers bound memory to the main agent when execution events match predefined conditions. Foresight uses an independent agent to predict potential queries, retrieve results in parallel, and store them in a cache, delivering matching and valid results when actual requests arrive. Hooks and Foresight achieve scores of 41.23% and 44.34% on MCP-Persona, respectively, while effectively reducing execution costs and latency and improving the efficiency of long-horizon agent execution.

AI Impact Assessments

(1 model)

Core Contribution

The paper argues that agent-memory research has focused on storage, update, and retrieval while neglecting *scheduling*. In the usual on-demand design, the main agent must notice that it needs memory, build a query, and block while retrieval runs. MemSidecar moves this work to a separate "sidecar" with two independently selectable modes:

Hooks: an LLM generates executable predicates bound to memory entries. When a query or pre-tool event matches, the entry is pushed to the agent without a memory query.

Foresight: a separate agent predicts likely queries from the task instruction and a topic directory, prefetches results into a task cache, and serves them when the main agent's actual query arrives. An LLM judges whether the actual and predicted queries match.

Both sit on a dual-layer store: a topic-organized knowledge graph (Global) and skills plus working memory (Workspace).

The conceptual framing is the main contribution. Moving memory access off the critical path, via event triggers and speculative prefetch, imports ideas from OS and systems design (read-ahead, interrupts/hooks). The idea is intuitive and plausibly useful. However, it is not a fundamental paradigm shift. FLARE, Self-RAG, and the broader agent-harness hooks ecosystem already occupy adjacent ground. The dual-layer memory is fairly conventional.

Methodological Rigor

Strengths:

Two benchmarks (MemoryArena, MCP-Persona).

Six baselines, split between memory systems (Mem0, A-Mem, LightMem) and harness policies.

A shared executor (GPT-5.4-mini) and a shared small memory model (Qwen3-8B).

Reporting of latency and token cost alongside accuracy.

Layer ablations.

A shared-backend comparison (Table B3) that partly isolates scheduling from storage.

A retrieval-intervention experiment showing that extra retrieval in baselines costs about 2x tokens and time while still trailing Foresight.

Stage-wise analysis, a case study, and appendix experiments on training the predictor and on joint scheduling.

Weaknesses:

No variance, seeds, or significance tests, although many differences are small on small benchmarks. MCP-Persona has 173 tasks and 724 checkpoints. MemoryArena complete-task SR is in the low single digits for shopping and travel, so differences of 1–2 points are likely noise.

Baseline fairness is uncertain. Baselines like OpenClaw, ZCode, and DSH are used in restricted subsets and "common executor" adaptations. MemSidecar likely has tuned integration (a six-entry budget, a 6,000-character delivery cap) that the baselines may not share.

The storage backend differs from the baselines in the main tables. The shared-backend table addresses this only for Mem0 and DSH, and the backend is described only briefly.

The Hooks mechanism is evaluated with simple matching (query content, action names and arguments). Its predicate-generation reliability, false-trigger rate, and Foresight's prediction accuracy and cache-hit rate are not reported. Such diagnostics are essential for validating the proposed mechanism.

Latency gains are confounded: the sidecar delivers less text and avoids main-agent query steps. Hooks' speed advantage may come from fewer retrieval turns rather than from parallelism. Foresight is slower than Hooks, which complicates the "parallel prefetch reduces latency" story.

Foresight's prediction budget is K=4 from the initial instruction, so it largely predicts early.

The appendix training experiment uses MemoryArena questions with canonical answers from "GPT-5.6-sol", which is obscure.

Several citations carry future-dated arXiv IDs, which makes verification impossible here.

Potential Impact

Practical relevance is high for production agent frameworks. Hooks resemble existing harness hook mechanisms, and speculative memory prefetch is directly implementable. The results suggest meaningful improvements: +4.7 to +7.8 points over the best baseline on MCP-Persona, with 25–45% lower latency. The "extra retrieval costs more but still loses" finding is a useful argument. The work could encourage treating memory scheduling as a first-class design axis and spawn follow-ups on learned prefetch policies, trigger synthesis, and cache validity. However, the contribution is a system design with many components (dual-layer memory, hooks, foresight, maintenance). Its generality beyond two benchmarks and one executor model is unproven, and it is unclear which components drive gains.

Timeliness

Agent memory and long-horizon agents are very active areas. Latency and cost of memory in agent loops are real bottlenecks, and the framing aligns well with the current interest in harnesses, MCP, and agent systems. The newness of both benchmarks means few strong baselines exist, so the paper is timely but also exposed to fast obsolescence.

Strengths

A clear, memorable problem framing (who schedules memory) with a systems flavor.

Two complementary, independently usable mechanisms.

Cost and latency reported alongside quality.

Fairly comprehensive appendix experiments: ablations, joint scheduling (Hooks+Foresight reaches 46.13%), and predictor training (SFT+RL 49.17%).

Consistent gains across categories.

Limitations

No variance or statistical testing, and low absolute performance on MemoryArena (travel SR under 5%).

Little mechanism-level analysis: hook precision, prediction hit rate, and cache-validity statistics.

Heavy dependence on LLM-based matching and predicate generation with a small 8B model; robustness is unclear.

Hooks require an LLM to guess future applicability conditions and fire once per task, which limits dynamic use.

Baselines are adapted and may not be tuned equally.

The dual-layer backend entangles memory quality with scheduling.

No code release is mentioned, although the appendix gives good algorithmic detail.

The writing is dense and abstract in places, with heavy notation for simple ideas.

Overall

This is a solid, timely systems-flavored agent paper with a useful framing and reasonably thorough experiments. The idea is intuitive rather than revolutionary, and the evidence lacks statistical rigor and mechanism diagnostics. I expect moderate-to-good citation impact within the agent-memory subfield.

Rating:6.5/ 10

Significance 6.5Rigor 5.5Novelty 6Clarity 6

Generated Nov 3, 2026

Comparison History (0)

No comparisons yet.

Scorecard· 16/16

6.5/10 impact

Timely, well-framed systems contribution with consistent gains in accuracy and latency on two benchmarks, but incremental in mechanism and lacking variance/diagnostics.