Library Drift: Diagnosing and Fixing a Silent Failure Mode in Self-Evolving LLM Skill Libraries

Paper 1 addresses a more fundamental and broadly applicable failure mode in LLM agents (premature commitment), with rigorous methodology including cross-model replication (3 models), cross-benchmark validation, mechanistic interpretability via hidden-state analysis, and honest reporting of limitations. The concept of representational commitment as an early diagnostic is novel and applicable across diverse agent architectures. Paper 2 tackles an important but narrower problem (skill library management) with a practical fix validated on a single benchmark. Paper 1's broader relevance to LLM reasoning, interpretability, and agent reliability gives it higher cross-field impact potential.

Paper 1 (Hera) addresses a broadly relevant and practical problem—efficient device-cloud coordination for LLM agents—with a rigorous two-stage training paradigm (imitation learning + reinforcement learning) evaluated across three diverse benchmarks. It achieves strong results (92.5% of cloud performance at 46.3% cloud usage), offering immediate practical value for deploying LLM agents at scale. Paper 2 identifies an interesting failure mode (library drift) but addresses a narrower problem within self-evolving skill libraries, evaluated on a single coding benchmark. Hera's broader applicability, methodological depth, and multi-domain evaluation give it higher potential impact.

Paper 1 addresses a fundamental, domain-agnostic failure mode in self-evolving LLM agents (library drift) and provides a reproducible fix with strong empirical gains. Its methodological innovation advances core AI agent architectures, offering broader impact across any field using autonomous AI, whereas Paper 2, despite its rigorous safety evaluation, is largely restricted to the specific domain of dentistry.

Paper 2 has higher impact potential: it identifies a broadly relevant, practically urgent failure mode in self-improving LLM agent systems (“library drift”), provides a clear causal trigger, introduces actionable diagnostics, and demonstrates a substantial, quantified fix with multiple ablations—strong methodological rigor and immediate real-world applicability. Its governance recipe and instrumentation can generalize across many agent/tooling frameworks, affecting reliability and long-horizon autonomy. Paper 1 offers a useful interpretability measurement framework, but it is narrower in application (spatial/recursive models) and its primary contribution is analytic rather than a widely deployable reliability improvement.

Paper 2 has higher estimated impact: it introduces a broadly applicable algorithmic framework (MEMOIR) for LLM-driven program/solver synthesis with explicit cross-branch knowledge transfer, validated across seven real combinatorial-optimization domains with strong gains in feasibility, quality, and—crucially—stability across runs. This targets high-value, real-world CO applications and is timely for LLM agents. Paper 1 is rigorous and valuable for maintaining self-evolving skill libraries, but is more niche (governance/diagnostics for a specific agent paradigm) and likely narrower in cross-field uptake.

Paper 1 introduces a fundamental theoretical framework (Generalized Turing Test) for comparing intelligence across arbitrary agents, which addresses a long-standing foundational question in AI evaluation. Its task- and dataset-agnostic nature gives it broad applicability across the entire field, with potential implications for training objectives, benchmarking, and philosophy of mind. Paper 2 addresses a specific, practical failure mode in skill libraries—important engineering work but narrower in scope. The GTT framework's potential to reshape how we think about evaluating and comparing intelligence gives it significantly broader and deeper scientific impact.

Paper 2 addresses a fundamental question about LLM training data composition and mathematical reasoning at scale (10T tokens), with findings that challenge a widely-held assumption (code improves general reasoning). Its insights about structured reasoning traces vs. executable code have broad implications for foundation model training across the industry. Paper 1, while methodologically rigorous, addresses a narrower problem (skill library management in self-evolving agents) with a smaller community of practitioners. Paper 2's mechanistic analysis via expert-activation patterns and practical data-centric optimization strategies give it wider applicability and timeliness.

Paper 2 addresses a critical and timely concern—ethical value alignment of LLMs in healthcare—with broad interdisciplinary impact spanning AI safety, medical ethics, and policy. Its framework for auditing value pluralism is novel and applicable across domains. The finding that models exhibit near-deterministic ethical stances risks 'deployment monoculture' has immediate policy implications as medical AI scales. Paper 1, while technically solid, addresses a narrower engineering problem (skill library management) relevant primarily to the LLM agent research community, limiting its breadth of impact.

AutoLLMResearch addresses a fundamental, high-stakes problem in LLM research—automating expensive experiment configuration—with a comprehensive framework (multi-fidelity environment with 1M+ GPU hours, structured training pipeline). Its breadth of impact is larger: it targets the core bottleneck of LLM scaling research affecting the entire field. Paper 2 identifies an important but narrower issue (library drift in skill libraries) with a focused fix. While methodologically sound, its scope is limited to self-evolving agent libraries. Paper 1's novelty in cross-fidelity extrapolation and practical resource savings gives it broader, more transformative potential.

Paper 1 introduces a novel approach to structure-based drug design by leveraging electron density as a conditioning signal, bridging computational and experimental structural biology with generative AI. This addresses a fundamental limitation in current SBDD methods and has direct real-world applications in pharmaceutical development. Paper 2 identifies and addresses an interesting failure mode in self-evolving LLM skill libraries, but its scope is narrower—focused on a specific engineering problem in agent systems. Paper 1's interdisciplinary nature (AI + structural biology + drug design), methodological novelty, and broader potential impact on drug discovery give it higher estimated scientific impact.