Echo: Learning from Experience Data via User-Driven Refinement
Hande Dong, Xiaoyun Liang, Jiarui Yu, Jiayi Lin, Changqing Ai, Feng Liu, Wenjun Zhang, Rongbi Wei
Abstract
Static "human data" faces inherent limitations: it is expensive to scale and bounded by the knowledge of its creators. Continuous learning from "experience data" - interactions between agents and their environments - promises to transcend these barriers. Today, the widespread deployment of AI agents grants us low-cost access to massive streams of such real-world experience. However, raw interaction logs are inherently noisy, filled with trial-and-error and low information density, rendering them inefficient for direct model training. We introduce Echo, a generalized framework designed to operationalize the transition from raw experience to learnable knowledge, effectively "echoing" environmental feedback back into the training loop for model optimization. In today's agent ecosystem, user refinement serves as a primary source of such feedback: driven by responsibility for the outcome, users rigorously transform flawed agent proposals into verified solutions. These user-driven refinement sequences inherently distill agents' crude attempts into high-quality training signals. Echo systematically harvests these signals to continuously align the agent with real-world needs. Large-scale validation in a production code completion environment confirms that Echo effectively harnesses this pipeline, breaking the static performance ceiling by increasing the acceptance rate from 25.7% to 35.7%.
AI Impact Assessments
(1 model)Scientific Impact Assessment: Echo: Learning from Experience Data via User-Driven Refinement
1. Core Contribution
Echo proposes a framework for learning from "experience data" — specifically, the discrepancy between an AI agent's initial proposals and user-refined final outcomes in production settings. The key insight is that users, as "accountable stakeholders," naturally correct flawed agent outputs into verified solutions, creating high-quality supervision signals. The framework is formalized as a three-stage pipeline: Experience Acquisition (capturing raw interaction streams), Knowledge Extraction (mining the gap between agent proposals C₁ and user-committed final states C_N), and Model Optimization (aligning the model to predict C_N directly from context C₀).
The concrete instantiation is in code auto-completion within Tencent's CodeBuddy product, where the system tracks code gaps between prefix and suffix anchors, monitors user edits until a contextual break, and extracts the final committed code as ground truth. A multi-stage data refinery pipeline handles truncation, quality filtering, distribution balancing, and PPL-based denoising.
2. Methodological Rigor
Strengths in evaluation design: The paper deliberately anchors evaluation in production metrics (Acceptance Rate, Generation Rate) rather than static benchmarks, which is appropriate for the industrial context and provides more convincing evidence of real-world utility. The five-month longitudinal evaluation with 10,000+ DAU is compelling.
Weaknesses in experimental rigor:
3. Potential Impact
The paper addresses a genuinely important problem: how to continuously improve AI agents from deployment-time interactions rather than static training data. The conceptual framing — users as accountable stakeholders who naturally produce ground-truth corrections — is intuitive and practically grounded.
Real-world applications: The framework is directly applicable to any AI copilot system where users refine agent outputs (code completion, writing assistants, design tools). The production deployment at Tencent scale provides credibility. The 10% absolute improvement in acceptance rate is commercially significant.
Broader influence: The "experience data" framing connects to important ongoing discussions about data scarcity for LLM training. However, the actual technical contribution is relatively straightforward — it essentially amounts to mining edit histories for SFT targets with careful data curation. The conceptual framework, while articulated with ambitious language, does not introduce fundamentally new technical machinery.
4. Timeliness & Relevance
The paper is highly timely. The question of how to learn from deployment interactions is central to the current AI agent ecosystem. Products like Cursor, GitHub Copilot, and Claude Code generate enormous interaction logs, and the industry desperately needs principled approaches to leverage this data. The paper's framing aligns well with the "Era of Experience" discussion (Silver & Sutton, 2025) and concerns about static data exhaustion.
5. Strengths & Limitations
Key Strengths:
Notable Limitations:
Additional Observations
The writing is clear but at times excessively promotional, using terms like "inexhaustible data engine," "strategic moat," and "paradigm shift" for what is essentially a well-executed data curation pipeline with SFT. The discussion section (6.3) speculates that experience data might surpass pre-training, which is unsupported by the evidence presented. The paper would benefit from more measured claims proportional to its actual experimental evidence.
Generated May 22, 2026
Comparison History (20)
Paper 1 addresses a highly timely and impactful problem: learning from experience data in deployed AI agents, a critical direction as agentic systems proliferate. It demonstrates strong real-world validation with a significant performance gain (25.7% to 35.7%) in production. Paper 2 offers valuable methodological rigor for IRT-based AI evaluation but has narrower scope and appeal, focusing on a specialized evaluation technique. Paper 1's framework has broader applicability across the rapidly growing agent ecosystem and directly tackles a fundamental scalability barrier, giving it greater potential for cross-field impact.
While Paper 1 offers a valuable framework for continuous AI model improvement in production, Paper 2 addresses a fundamental, systemic crisis across the scientific enterprise: reproducibility. By demonstrating that LLMs can automate data reanalysis and recover effect sizes at scale—matching or exceeding human reanalysts—Paper 2 has massive potential to revolutionize peer review, meta-science, and research integrity across multiple empirical disciplines. This broad, foundational impact on the scientific method itself outweighs the industry-focused, domain-specific AI optimization presented in Paper 1.
Paper 2 addresses test-time scaling, currently one of the most critical frontiers in LLM reasoning. By introducing stochastic backtracking over historical prefixes, it provides a novel, methodologically rigorous solution to premature commitment in search. This fundamentally improves the compute-accuracy trade-off, offering broad applicability across reasoning tasks. While Paper 1 presents a highly practical approach to continuous learning from user data, Paper 2's algorithmic innovation in reasoning search strategies is likely to have a more widespread and immediate theoretical and practical impact across the AI research community.
Paper 2 (Echo) demonstrates higher potential scientific impact due to its practical applicability and validated real-world results in a production environment, showing a significant 10% improvement in code completion acceptance rates. It addresses the fundamental and timely challenge of continuous learning from deployment data, which is broadly applicable across AI agent systems. While Paper 1 makes a solid contribution to CoT faithfulness detection using mechanistic interpretability, it addresses a more niche problem. Paper 2's framework for leveraging user refinement as training signals has broader implications for how AI systems are trained and improved at scale.
Paper 1 identifies a novel, fundamental failure mode in agentic AI (temporal memory contamination) and introduces a rigorous evaluation protocol for longitudinal safety. As long-term memory becomes standard in LLMs, establishing how accumulated context degrades safety will have profound implications across AI alignment, safety evaluations, and architecture design, offering broader scientific impact than Paper 2's application-focused data pipeline.
Paper 1 offers higher scientific impact by identifying a novel, fundamental vulnerability in AI agents: temporal memory contamination. While Paper 2 provides a highly practical, production-validated framework for continuous learning from user feedback, it builds on established paradigms of interaction-based alignment. Paper 1 pioneers a new longitudinal evaluation paradigm for AI safety, demonstrating that risks compound over time across unrelated tasks. Its rigorous trigger-probe protocol and early detection mechanism provide foundational tools for future research in secure, long-horizon autonomous agents, making its conceptual contributions more broadly impactful.
Paper 1 addresses the critical AI bottleneck of data scaling by presenting a scalable, continuous learning framework from user interactions. Its validation in a large-scale production environment demonstrates immediate, widespread real-world utility across the rapidly growing domain of LLM agents, offering broader impact than Paper 2's robotics-specific optimizations.
Paper 2 addresses a fundamental bottleneck in AI training—the reliance on expensive, static human data—by introducing a scalable framework for continuous learning from user-driven refinements. This approach has massive breadth of impact across virtually all interactive AI agents. Furthermore, its validation in a large-scale production environment demonstrating a 10% absolute increase in acceptance rates highlights exceptional real-world applicability and timeliness. While Paper 1 offers a rigorous and novel architectural improvement for robotic control, Paper 2's paradigm shift toward continuous experience-based learning has a wider potential footprint across the broader AI ecosystem.
Paper 1 (LGBO) demonstrates higher scientific impact potential due to: (1) Novel theoretical framework integrating LLMs into Bayesian Optimization with formal convergence guarantees, (2) Broad applicability across multiple scientific domains (physics, chemistry, biology, materials science), (3) Validated in both dry benchmarks and wet-lab experiments showing significant efficiency gains, (4) Addresses fundamental challenges in scientific discovery (costly experiments, cold-start, high dimensionality). Paper 2 (Echo) addresses an important but narrower problem in code completion agent training. While practically valuable with strong production results, its impact is more domain-specific compared to LGBO's cross-disciplinary scientific optimization framework.
Paper 2 introduces a novel framework (LGBO) that integrates LLM reasoning into Bayesian Optimization with theoretical guarantees and demonstrates impact across multiple scientific domains including wet-lab validation. Its breadth of impact spans physics, chemistry, biology, and materials science, addressing a fundamental challenge in scientific discovery. Paper 1, while practically valuable with strong production results in code completion, is more narrowly focused on a single application domain. Paper 2's theoretical contributions, cross-disciplinary applicability, and real experimental validation suggest broader and deeper scientific impact.