EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learning

Paper 1 addresses a fundamental and underexplored safety gap in autonomous agent evaluation—compliance bias and the absence of abstention benchmarks. It introduces a novel taxonomy, concrete evaluation protocols, and preliminary empirical results across 144 scenarios. This has broad impact across AI safety, agent deployment, and benchmark design communities. Paper 2, while technically strong in co-evolving training harnesses, represents a more incremental advance in RL training methodology. Paper 1's contribution to safety evaluation frameworks is more timely and broadly impactful as autonomous agents are increasingly deployed in real-world settings.

Paper 2 likely has higher impact: it proposes a new autonomous RL training paradigm (co-evolving policy + training harness), targets a timely bottleneck in agentic RL (reward misalignment/hidden failure modes), and reports competitive or better results on high-value benchmarks including long-horizon software engineering—high real-world applicability and broad relevance to RL, agent design, and LLM training pipelines. Paper 1 is methodologically careful and insightful for mechanistic interpretability, but its applications are more indirect and its influence may be narrower than a training framework that can materially improve autonomous agents.

Paper 2 likely has higher impact: it introduces a simple, broadly applicable architectural primitive (score-level SSM-attention fusion) that plugs into standard SDPA without custom kernels or recurrent state, making adoption easy across model builders and hardware stacks. The method targets a timely, central bottleneck (long-context retrieval + prioritization) and reports strong gains on established benchmarks and retrieval metrics, suggesting cross-field relevance (NLP, efficient transformers, systems). Paper 1 is novel for autonomous RL training workflows, but its impact may be narrower and more dependent on complex evaluation setups and toolchains.

Paper 2 proposes a fundamental architectural shift in LLM agents by eliminating fixed orchestrators in favor of self-programmed execution. This introduces a highly novel paradigm and a custom language (Spell), offering broader theoretical implications and the potential to spawn a new subfield of self-orchestrating agents. While Paper 1 presents a strong, rigorous automated training framework, Paper 2's foundational innovation in agent design gives it a higher potential for disruptive, long-term scientific impact across AI research.

Paper 1 introduces a novel, general-purpose framework for autonomous LLM training (co-evolving policies and training harnesses) that applies broadly across math, coding, and software engineering. Its fundamental methodological advancement in agentic RL offers greater potential for widespread adoption and transformative impact across the AI field compared to Paper 2, which focuses on a domain-specific evaluation benchmark for chemistry.

EvoTrainer introduces a fundamentally novel paradigm shift—co-evolving both LLM policies and training harnesses—addressing a core limitation in autonomous RL training. Its breadth across mathematical reasoning, code generation, and software engineering demonstrates wide applicability. The concept of moving beyond static recipe search toward joint evolution is a more transformative contribution with broader implications for the entire LLM training ecosystem. StepFinder, while useful for failure attribution in multi-agent systems, addresses a narrower diagnostic problem with incremental improvements over existing methods.

Paper 1 offers a profound breakthrough with broader scientific implications by addressing the black-box limitation of modern AI. By autonomously discovering interpretable, extrapolatable governing equations, it impacts virtually every empirical science discipline. Its demonstrated reduction in extrapolation error by six orders of magnitude and massive parameter compression represents a monumental leap over standard deep learning. While Paper 2 presents significant, timely advancements in LLM training and agentic RL, Paper 1's fundamental contribution to explainable AI-driven scientific discovery gives it a wider, more transformative potential across the broader scientific community.

Paper 1 has higher potential scientific impact due to its broad, unifying theoretical contribution linking variational inference/free energy, stochastic games, and thermodynamics, with formal equivalence results (stationary points ↔ approximate Nash equilibria) and an interpretable higher-order synergy construct (free-energy Harsanyi dividend). Its claims are cross-disciplinary (neuroscience, biology, AI, economics) and include falsifiable predictions validated across domains, suggesting durable conceptual influence. Paper 2 is timely and practically valuable for LLM RL, but is more field-specific and likely to be superseded by rapid iteration in methods, with narrower foundational reach.

Paper 1 addresses a fundamental bottleneck in the physical sciences (materials and drug discovery) by bridging generative AI with physical simulations. Its tenfold acceleration in discovering diverse molecular and crystal structures offers profound real-world applications in developing new materials and pharmaceuticals. While Paper 2 presents an innovative LLM training methodology, Paper 1 demonstrates higher cross-disciplinary impact and broader potential for tangible scientific breakthroughs in chemistry and physics.

Paper 2 presents a foundation model for wearable health pretrained on an unprecedented scale (1 trillion minutes, 5 million participants), addressing a fundamental challenge in digital health. Its breadth of impact spans 35 health prediction tasks across multiple domains, demonstrates few-shot learning capabilities, and integrates with a clinician-validated Personal Health Agent. The combination of massive-scale pretraining, clinical validation, and practical deployment potential gives it broader real-world impact across healthcare, AI, and wearable technology. Paper 1, while innovative in co-evolving RL training harnesses, addresses a narrower ML methodology problem.

Paper 2 (ReClaim) presents a healthcare foundation model trained on 43.8 billion medical events from 200M+ patients, demonstrating broad utility across disease prediction, expenditure forecasting, and causal inference (target trial emulation). Its impact spans clinical research, health economics, regulatory science, and AI methodology. The scale of validation (1,000+ prediction tasks, external validation, prospective evaluation) and direct relevance to healthcare decision-making give it enormous real-world applicability. Paper 1 (EvoTrainer) is innovative in co-evolving training harnesses with LLM policies, but its impact is more narrowly focused on ML training methodology, a less broadly consequential domain.

Paper 1 likely has higher scientific impact: it offers a broad, timely, and methodologically substantial empirical critique of LLM-based “AI scientist” agents across eight domains and 25,000+ runs, identifying systematic epistemic failures that standard outcome metrics miss. The findings directly affect how the community evaluates and trusts autonomous scientific discovery, with implications spanning AI, HCI, scientific methodology, and research governance. Paper 2 is novel and practically useful for agentic RL training, but its impact is more concentrated within ML systems/optimization, whereas Paper 1 challenges foundational assumptions about autonomous science and evaluation paradigms.

While Paper 1 presents a strong methodological advancement in autonomous LLM training, Paper 2 demonstrates a groundbreaking milestone: the first end-to-end autonomous scientific discovery by an AI agent in a real physical system. By discovering and experimentally validating a previously unreported physical mechanism (optical bilinear interaction) with direct implications for optical computing, Paper 2 showcases unprecedented real-world application and represents a major paradigm shift in how scientific research can be conducted.

HealthFormer addresses a fundamental challenge in medicine—personalized health forecasting and intervention simulation—with a single generative model trained on deeply phenotyped longitudinal data across 667 measurements and 7 domains. Its ability to simulate clinical interventions in silico, validated against 41 randomized trial comparisons, has transformative potential for precision medicine, drug development, and clinical decision-making. The breadth of impact (transferring to 4 independent cohorts, outperforming established clinical risk scores across 27/30 endpoints) and the concept of 'clinical digital twins' represent a paradigm shift with far wider real-world applicability than EvoTrainer's RL training framework improvements.

Paper 2 likely has higher scientific impact: it introduces a broadly applicable multimodal generative foundation model plus a new aligned dataset spanning major biomolecular modalities, enabling advances in prediction and design across RNA/protein biology and potentially drug discovery. Its applications (splicing, clinical variant editing suggestions, protein binder design, assay-context modeling) are timely and high-stakes, with impact across genomics, structural biology, and therapeutics. Paper 1 is innovative for autonomous RL training infrastructure, but its impact is more confined to LLM training methodology and may depend on broader adoption and reproducibility of the co-evolution framework.

Paper 2 proposes a foundational paradigm shift in agentic RL by co-evolving policies and training harnesses. While Paper 1 identifies a critical safety vulnerability, Paper 2's framework for autonomous self-improvement addresses a core bottleneck in LLM training pipelines. Its ability to outperform human-engineered RL across complex domains (math, coding, SWE) suggests a broader, cascading impact on the fundamental development and training methodologies of future autonomous agents.

EvoTrainer introduces a fundamentally novel paradigm for autonomous LLM training—co-evolving both policies and training harnesses—addressing a core limitation in agentic RL. It demonstrates strong results across multiple challenging domains (math reasoning, code generation, software engineering) and proposes a generalizable framework that could reshape how LLM training is conducted. Paper 1 contributes a valuable dataset for deepfake detection and human-agent interaction research, but is more incremental in scope. Paper 2's methodological innovation and broad applicability across the rapidly growing LLM/RL field give it higher potential impact.

Paper 1 has higher impact potential due to its broader, more general contribution: co-evolving both agent policies and the training harness addresses a core bottleneck in autonomous RL for LLM agents and is applicable across many domains (math, coding, long-horizon SWE, beyond). It targets timelier, widely relevant needs in scalable agent training and evaluation robustness, with clear real-world leverage for improving autonomous systems. Paper 2 is novel and valuable but is narrower in scope (formal proof refactoring in Lean) and likely to impact a smaller community, despite strong methodological motivation.

Paper 1 has higher potential impact due to a more novel and broadly applicable paradigm: co-evolving both LLM policies and the training harness/diagnostics for agentic RL, addressing a key bottleneck in autonomous LLM training. It is timely and relevant to rapidly advancing LLM and RL research, with demonstrated gains on multiple challenging domains (math, code, long-horizon SWE) and emphasis on methodological safeguards (diagnostics, backtesting, avoiding invalid reward hacks). Paper 2 is narrower in scope (game enemy morphology PCG) with more limited cross-field and real-world impact.

EvoTrainer introduces a fundamentally new paradigm for autonomous LLM training by co-evolving both policies and training harnesses, addressing a deeper structural limitation in RL-based LLM training. Its impact spans multiple domains (math reasoning, code generation, software engineering) and proposes a generalizable framework that could reshape how LLM training is conducted. Paper 1, while practical and well-executed, addresses a narrower optimization problem (token cost reduction via prompt rewriting) with incremental engineering contributions rather than a conceptual shift in methodology.