RSIAgent: Autonomous Exploration for Recursive Self-improvement in New Environments
Sibo Zhu, Shicheng Fan, Xinyue Wang, Wenyi Wu, Kun Zhou, Biwei Huang
Timely, well-engineered agent-memory framework with useful case studies, but crowded conceptual space and a headline claim undermined by selective task-based aggregation.
Abstract
Digital agents must often adapt to new environments whose interfaces, tools, and failure modes are not fully captured by pretrained models. We introduce \textbf{RSIAgent}, a training-free multi-agent framework for recursive self-improvement through autonomous memory construction. RSIAgent coordinates curriculum, actor, and verifier agents to continually explore the environment, validate outcomes, and retain environment-specific knowledge, including reusable causal relationships between actions, conditions, and consequences. It further adopts a \textbf{broad-then-deep} exploration strategy, combining parallel broad recursive self-exploration for discovering diverse environment structures with focused deep self-exploration for uncovering hard cases, hidden constraints, boundary conditions, and previously unknown causal dependencies. The resulting memory is frozen and can be directly reused for downstream tasks without updating model parameters. Experiments on OSWorld-v2 and Agent's Last Exam show that RSIAgent substantially improves strong open-source models, enabling Kimi-K3 and GLM-5.3 to outperform frontier closed-source models including GPT-6.
AI Impact Assessments
(1 model)Scientific Impact Assessment
Core Contribution. RSIAgent proposes a training-free, multi-agent framework for adapting digital/computer-use agents to unfamiliar software environments without updating model weights. The system decomposes self-improvement into three roles — a curriculum agent (what to explore), an actor agent with evolvable memory (how to act), and a verifier agent grounded in environment feedback (whether it worked) — coordinated in a recursive loop. Its central design idea is a broad-then-deep exploration schedule: parallel Broad Recursive Self-exploration (BRS) to build coverage, followed by sequential Deep Recursive Self-exploration (DRS) to probe boundary conditions and hard cases. The accumulated memory is then frozen and reused at test time. This is a sensible and coherent synthesis, but conceptually it sits in a very crowded space (Voyager, Reflexion, EchoTrail-GUI, ZhuLong, CoEvoSkills, HyMEM/HySEM, ACE, AutoManual, and the authors' own prior HyMEM work). The "causal relationship" framing is repeatedly invoked but never formalized or evaluated as causal discovery — it functions rhetorically rather than technically, which weakens the claimed novelty of that angle.
Methodological Rigor. This is the paper's weakest dimension, and notably the authors are unusually transparent about it. The headline claim — that RSIAgent lets open-source Kimi-K3/GLM-5.3 surpass frontier closed models (GPT-6 Astra, Claude Opus 5) — rests on an aggregation procedure that is not a matched-protocol comparison. RSI exploration is selectively directed only at tasks "with remaining room for improvement," and for the many tasks that receive no exploration, the *baseline* scores are simply retained in the aggregate. Only 41 of 82 OSWorld tasks and 19 of 67 ALE tasks have genuine RSI entries; the rest carry baseline numbers. This design structurally inflates aggregate gains and cannot isolate the framework's contribution. On ALE the improvement is marginal (83.75→84.82 partial) and binary accuracy actually *trails* GPT-6 (50.75 vs 52.24). Ablations use only four tasks; failure-mode analysis rests on tiny case audits (e.g., 3/4, 2/6). Comparison numbers are copied from heterogeneous leaderboards/tech reports with unmatched harnesses and budgets. The appendices honestly flag "provisional aggregates," retained regressions, corrected grades, and public-label transfers — good scientific hygiene, but this candor also reveals that the top-line claims are only weakly supported.
Potential Impact. Computer-use / GUI agents and training-free memory-based adaptation are extremely active areas with clear commercial pull (agentic automation, RPA, autonomous software operation). The general recipe — curriculum-driven autonomous exploration + independent verification + consolidated reusable memory — is intuitive, modular, and likely to be adopted or referenced by practitioners building agent harnesses. The extensive case studies (video editing, CAD reconstruction, REAPER audio, railway booking) are genuinely useful as qualitative demonstrations of how memory accumulates and transfers, and provide reusable engineering patterns (checkpoint-protected verification, parallel-work/sequential-update reconciliation, evidence-grounded PASS/FAIL/UNVERIFIED verdicts). However, given how many near-identical concurrent systems exist, the paper is more likely to be one citation among many in this subfield than a defining reference.
Timeliness & Relevance. Very high. Adapting pretrained agents to private, changing, or specialized environments without retraining is a recognized bottleneck, and the paper directly targets it on current flagship benchmarks (OSWorld 2.0, Agents' Last Exam, GameCraft-Bench). The framing against the "pretraining-then-posttraining" paradigm is apt for the moment.
Strengths.
Limitations / Weaknesses.
Other observations. The paper's evidence is best read as an existence proof that memory-guided self-exploration produces meaningful per-task improvements on hard, long-horizon tasks — which it demonstrates convincingly at the case level — rather than as a rigorous benchmark-topping result. Its lasting value is more methodological/architectural than empirical. Reproducibility is moderate: prompts and algorithms are specified and code is claimed, but the reliance on specific frontier models and non-deterministic exploration would make exact replication difficult.
Overall, this is a competent, timely systems contribution in a fast-moving subfield, with a sound architecture and useful engineering patterns, but with evaluation methodology that does not robustly support its strongest claims and limited conceptual novelty over closely related concurrent work.
Generated Sep 15, 2026
Comparison History (0)
Timely, well-engineered agent-memory framework with useful case studies, but crowded conceptual space and a headline claim undermined by selective task-based aggregation.