SkillOpt: Executive Strategy for Self-Evolving Agent Skills

Paper 1 has higher impact potential: it proposes a concrete, novel, and rigorously controlled optimization framework (text-space “optimizer” with held-out validation gating, stability mechanisms, and zero inference-time overhead) and reports broad empirical wins across many models/benchmarks/harnesses with transfer evidence—suggesting actionable, reproducible improvements for agent reliability and deployment. Paper 2 is a valuable, timely survey that may shape terminology and evaluation norms, but it contributes less direct methodological innovation and lacks empirical validation of a new system, typically yielding lower immediate scientific/engineering impact than a strong, generalizable technique.

Paper 2 has higher potential impact due to a more clearly novel and generalizable contribution: a controllable, validation-gated “text-space optimizer” for agent skills that behaves like an optimizer with stability mechanisms (budgets, rejected-edit buffer, slow/meta updates). It demonstrates broad, rigorous evaluation (6 benchmarks, 7 models, 3 harnesses, 52 cells) plus transfer across models and environments, supporting robustness and real-world deployability with zero added inference cost. Paper 1 is timely and useful for efficiency, but is closer to re-factoring planning/self-regulation within CoT and may be less broadly transformative.

SkillOpt demonstrates broader empirical impact across 52 evaluation cells spanning six benchmarks, seven models, and three execution harnesses, consistently outperforming all competitors. Its framing of text-space skill optimization as analogous to weight-space optimization introduces a novel paradigm with wide applicability to any LLM agent. The transferability results across models and environments further strengthen its practical impact. While EVE-Agent addresses an important trustworthiness problem in self-evolving search agents, its scope is narrower (search agents only), and its contribution is more incremental—adding an evidence verification step to existing proposer-solver frameworks.

Paper 1 has higher potential impact because it targets a foundational, cross-cutting bottleneck—reproducibility and evaluation integrity in agent research—affecting most subfields using agent rollouts. The rollout-card standard plus reference implementation can become infrastructure adopted broadly, enabling reanalysis, audits, and fairer comparisons, with immediate relevance as agent benchmarks proliferate. Its empirical audit of repositories and demonstrated score/ranking sensitivity to reporting rules strengthens methodological rigor and urgency. Paper 2 is promising and practically useful, but its impact is narrower (skill optimization technique) and more contingent on specific agent setups.

Paper 1 introduces causal inference principles to language agent tool use, addressing a fundamental limitation in how agents learn from observational data. Bridging causality and LLM agents opens a highly novel and rigorous research direction for agent safety and reliability. While Paper 2 offers excellent empirical results and a strong methodological approach to skill optimization, Paper 1's conceptual leap of treating tool calls as causal interventions provides a deeper theoretical contribution with broader long-term scientific impact across AI and decision-making.

Paper 2 (SkillOpt) likely has higher impact due to broader applicability and stronger empirical evidence. It proposes a reproducible, optimizer-like framework for improving agent “skills” with controlled edits and strict validation, yielding large, consistent gains across many models, benchmarks, and execution harnesses, plus demonstrated transfer. This suggests immediate real-world deployment value for agent reliability and performance. Paper 1 is novel conceptually (environment synthesis for RLVR) but shows modest gains on a narrower setting and may be harder to generalize; its impact is promising but less immediately broad.

Paper 2 demonstrates AI solving genuinely open mathematical problems (9 Erdős problems, 44 OEIS conjectures) with formal verification, representing a landmark achievement in AI-for-mathematics. This has transformative implications for mathematical research methodology and is already being deployed across multiple mathematical subfields. While Paper 1 presents a solid engineering contribution for optimizing agent skills with impressive benchmarks, it is ultimately an incremental advance in prompt/skill optimization. Paper 2's direct scientific discoveries and paradigm-shifting potential for mathematics research give it substantially higher impact.

Paper 2 likely has higher scientific impact due to a broadly applicable, optimizer-like framework for improving agent skills with strong empirical breadth (six benchmarks, seven models, three harnesses, 52 cells), clear methodological controls (held-out validation acceptance, stability mechanisms), and immediate real-world applicability to agent performance without added inference cost. Its “text-space optimization” paradigm can influence LLM/agent training, tool-use, and deployment workflows across domains. Paper 1 is novel and timely for human–machine discrimination, but its impact may be narrower (CAPTCHA-style evaluation and process-feature specification bottlenecks limit generalization).

SkillOpt presents a concrete, empirically validated method with extensive benchmarking across 52 evaluation cells, demonstrating consistent improvements. It introduces a novel framing of skill optimization analogous to weight-space training, with rigorous methodology including validation-based acceptance criteria and ablations. Paper 2 (Foundation Protocol) proposes a coordination framework but is primarily architectural/conceptual without empirical validation of its claims. While Paper 2 addresses an important emerging problem, its lack of concrete experimental results and its broad, position-paper-like nature limits near-term scientific impact compared to SkillOpt's reproducible, measurable contributions.

Paper 2 likely has higher scientific impact: it proposes a concrete, optimizer-like framework (SkillOpt) for reproducible, controllable skill improvement with validation-gated edits, and reports extensive empirical gains across many models/benchmarks/harnesses plus transfer. This combination of methodological specificity, measurable performance improvements, and near-term deployability makes it broadly useful for agent training and tooling. Paper 1 offers important conceptual guidance for benchmark design and interpretation, but is less directly actionable as a new method and may yield slower, more diffuse downstream adoption compared to a strong, general-purpose optimization technique.

Paper 2 has higher estimated impact due to broader applicability and timeliness: a systematic, controllable “text-space optimizer” for agent skills can improve many existing and future agentic systems without changing base model weights, and transfers across models/harnesses. Its methodology (held-out validation gating, bounded edits, learning-rate budget, stability mechanisms) resembles reproducible optimization and is likely to influence practice across AI agents, software engineering, and RL-style evaluation. Paper 1 is a strong, rigorous architectural advance for linear attention/sequence models, but its impact is narrower to efficient long-context model design.

SkillOpt introduces a novel and practically impactful framework—treating agent skills as trainable text-space parameters with optimizer discipline (learning rate, validation, edit buffers). It demonstrates broad empirical impact across 52 evaluation cells, multiple models, and execution harnesses, with large accuracy gains (+19-25 points). Its transferability across models and environments enhances real-world applicability. While GENSTRAT contributes a useful benchmarking methodology for strategic reasoning, SkillOpt addresses a more fundamental problem in AI agent development with a generalizable optimization paradigm that could influence how agent skills are developed across the field.

SkillOpt demonstrates broader impact with its systematic text-space optimizer for agent skills, showing improvements across 52 evaluation cells spanning six benchmarks, seven models, and three execution harnesses. Its conceptual contribution—treating skills as optimizable external state with learning-rate budgets and validation-based acceptance—introduces a novel paradigm analogous to weight-space optimization. The transfer results across models, environments, and tasks further strengthen its generalizability. Co-ReAct contributes meaningful step-level rubric guidance for ReAct agents but addresses a narrower problem scope with fewer benchmarks and less demonstrated generalization.

Paper 1 introduces a highly innovative framework that applies deep learning optimization principles to text-space agent skills. Its extensive evaluation across multiple benchmarks, models, and execution environments demonstrates remarkable robustness and significant performance gains. This systematic approach to skill optimization has higher potential for widespread adoption and broad methodological impact compared to Paper 2's narrower focus on library hygiene and lifecycle management.

Paper 1 introduces a highly novel, text-space optimization algorithm for agent skills inspired by deep learning optimizers. Its systematic approach yields massive performance gains across diverse benchmarks and execution harnesses. By proving that text-based skills can be rigorously optimized and transferred across model scales, it fundamentally advances agent training methodology, promising broad and immediate real-world applications.

SkillOpt introduces a fundamentally novel paradigm—treating agent skills as optimizable external state analogous to weight-space optimization—with rigorous methodology (52 evaluation cells, 7 models, 3 harnesses, ablations, transfer experiments). Its broad applicability across any LLM agent system, strong empirical gains (+19-25 points), and transferability across models and environments suggest wide adoption potential. Paper 1, while impressive as an engineering contribution to visualization, is more domain-specific and represents incremental progress in applying agentic AI to a narrower application area rather than introducing a general-purpose methodology.

SkillOpt introduces a fundamentally new paradigm—treating agent skills as trainable text-space parameters with optimizer-like discipline—which is highly novel and broadly applicable. Its comprehensive evaluation across 52 cells (6 benchmarks, 7 models, 3 harnesses) with consistent improvements and strong transfer results demonstrates exceptional methodological rigor and generalizability. The practical impact is substantial: zero inference-time overhead and compatibility with multiple execution environments. Paper 2 addresses an important but narrower problem (multimodal knowledge editing generalization) with incremental advances. SkillOpt's breadth of impact across the rapidly growing AI agents field gives it significantly higher potential impact.