Structured Scaling of AI Discovery Across Diverse Scientific Domains
Haotian Ye, Haowei Lin, Jingyi Tang, Yizhen Luo, Rahul Thapa, Caiyin Yang, Chang Su, Rui Yang
Broad, open, reproducible demonstration that simple structured scaling with a small open model matches or beats complex proprietary-model discovery systems, but many gains are marginal and the algorithmic novelty is modest.
Abstract
Scientific discovery often requires many cycles of proposing, testing, and refining candidate solutions. Language models can increasingly participate in these loops, but simply generating more attempts does not ensure progress: parallel searches may duplicate one another and iterative refinement may become trapped in poor directions. The central challenge is therefore not only to scale AI-driven discovery, but to structure that scaling so that evaluation signals compound over time. Here we introduce SimpleTES (Simple Test-time Evaluation-driven Scaling), a framework that focuses on the structured scaling of AI discovery loops, organizing evaluator queries across independent trajectories, iterative refinement, local candidate selection, and the selective reuse of evaluated histories. Drawing on structural features of scientific communities, SimpleTES uses a single open-source GPT-OSS model to establish new state-of-the-art solutions across 28 open-ended problems in diverse scientific domains ranging from quantum physics and astronomy to biology, AI, and mathematics. These include a 24.5% reduction in quantum circuit compilation overhead, up to 23% lower propulsive cost for deep-space trajectories, a 2.17x faster lasso-path solver, an 8.5% lower-error whole-brain neural-activity predictor, the fastest reported TriMul kernel, and new mathematical constructions beyond prior human or AI records. We further post-train the model for long-horizon discovery by assigning each attempt the final outcome of the trajectory it helped produce. This improves performance on both training and held-out mathematics problems, further advancing the frontier. Together, these results establish structured scaling as a general mechanism for advancing AI scientific discovery.
AI Impact Assessments
(3 models)Core Contribution
SimpleTES is a deliberately minimal test-time framework for evaluator-driven discovery. It allocates a budget N = C×L×K across C independent trajectories, L refinement steps, and K candidates per step, with only the best of K committed to the history. A proposal constructor Φ selects history nodes with a graph-based PUCT variant (RPUCG) and adds failure summaries. The paper's claim is that how evaluator queries are structured matters more than elaborate agentic machinery. A single open model, gpt-oss-120b, with one fixed configuration and only the evaluator changed, reports SOTA on 28 problems: quantum compilation (SWAP reduction, zoned neutral-atom placement), astrodynamics, GPU kernels (TriMul), SLDBench scaling laws, ZAPBench forecasting, a lasso path solver (2.17× over glmnet), scRNA-seq denoising, and several math constructions (Erdős overlap, autocorrelation inequalities, Sum–Difference). A second contribution is trajectory-level post-training: every attempt is credited with its trajectory's final score via rejection fine-tuning. This improves top-R% trajectory scores on four training tasks and four held-out math tasks, and yields a new Sum–Difference record of 1.144887.
Methodological Rigor
Strengths:
Weaknesses:
Potential Impact
The practical message is strong. A cheap open model plus structured search matches or beats proprietary-model systems across many domains. The result is a reusable, simple baseline and an open codebase for evaluator-driven discovery. Domain outputs such as the quantum routing/placement policies, the TriMul kernel (portable across A100/H100/MI300), and the geometry-aware lasso solver are of practical interest. The reward-hacking analysis is a useful warning that will inform evaluator design. The trajectory-credit post-training idea is simple and plausible, though it needs larger-scale validation. Limitations to adoption are the need for cheap, automatic, exploit-resistant evaluators, and weak applicability to sparse-reward problems such as theorem proving or wet-lab experiments.
Timeliness
This is highly timely. AlphaEvolve-style systems, OpenEvolve, ShinkaEvolve, ThetaEvolve, and TTT-Discover form a crowded and fast-moving area. A paper that argues for simplicity, reproducibility with open weights, and broad cross-domain validation fits a real need. It will probably be cited as a strong open baseline, though the field is likely to leapfrog it quickly as frontier models improve.
Strengths
Limitations
Overall
A solid, useful, well-engineered systems/empirical paper whose value lies in breadth, open-source accessibility, and clear framing rather than a deep conceptual breakthrough. Expected impact is moderate to high within the AI-for-discovery community, with limited transformative effect.
Generated Oct 8, 2026
Comparison History (286)
Paper 1 likely has higher impact due to demonstrated cross-domain, state-of-the-art empirical results on 21 scientific problems and clear real-world wins (e.g., >2× LASSO speedup, improved quantum routing, new combinatorial constructions). Its framework is broadly applicable to LLM-driven scientific workflows and produces reusable training data (trajectories) that improve generalization, increasing downstream adoption potential. Paper 2 is highly rigorous and clarifies discrete diffusion theory, but its impact is more specialized to generative modeling research and less directly tied to broad, immediate applications.
Paper 2 has higher estimated scientific impact due to its broader, cross-domain framework for scaling evaluation-driven discovery loops across 21 problems in six domains, with demonstrated state-of-the-art results and concrete improvements (e.g., faster LASSO, better quantum routing, new combinatorial constructions). Its methodology targets a general bottleneck—how to scale evaluation/feedback loops—which is timely and widely applicable to LLM-based scientific workflows, and it adds a learning component via trajectory supervision with generalization. Paper 1 is strong and novel but more domain-specific to porous media/materials design.
Paper 1 addresses a timely, broadly impactful problem—scaling evaluation-driven loops for LLM scientific discovery—and demonstrates concrete, verifiable state-of-the-art results across 21 problems in six domains, including a 2x LASSO speedup and novel mathematical constructions. Its generality and demonstrated empirical gains suggest wide cross-field impact. Paper 2 tackles an important copyright detection problem with elegant theoretical formalization, but remains largely theoretical with less demonstrated empirical validation and narrower applicability. Paper 1's combination of methodological rigor, breadth, and tangible discoveries gives it stronger potential impact.
Paper 1 has higher potential impact: it proposes a general, scalable framework (SimpleTES) for evaluation-driven discovery loops and demonstrates state-of-the-art gains across 21 problems in six scientific domains, including novel algorithmic and combinatorial discoveries. This breadth, cross-domain applicability, and link to trajectory-based post-training suggest broad downstream influence on LLM-based scientific methods. Paper 2 is methodologically solid and timely for efficient diffusion transformer deployment, but its contribution is more specialized to PTQ engineering for DiTs, with narrower cross-field reach.
Paper 1 introduces a framework that accelerates scientific discovery across multiple disciplines, yielding concrete breakthroughs in math and algorithms (e.g., speeding up LASSO, improving quantum routing). Its interdisciplinary applicability and proven success in uncovering novel scientific results offer a much broader and more profound potential impact than Paper 2. While Paper 2 provides a highly valuable and rigorous system-level optimization for long-context LLM inference, its impact is largely confined to AI engineering, whereas Paper 1 pushes the fundamental boundaries of multiple scientific fields.
Paper 2 has higher potential scientific impact due to broader cross-domain relevance and stronger demonstrated external scientific/algorithmic advances. It proposes a general evaluation-driven scaling framework (SimpleTES) and validates it across 21 problems in six domains, including concrete, verifiable improvements (e.g., faster LASSO, better quantum routing, new combinatorial constructions). This breadth plus direct scientific discovery outcomes suggests wider uptake beyond LLM post-training. Paper 1 is methodologically solid and has major industrial impact, but its core contribution is more specialized to evaluator/policy co-training in LLM alignment.
While Paper 2 offers significant methodological advancements in RLVR optimization, Paper 1 demonstrates broader and more immediate scientific impact. By formally scaling evaluation-driven discovery loops, Paper 1 solves concrete problems across six distinct domains, including accelerating LASSO, improving quantum circuit routing, and finding novel mathematical constructions. This cross-disciplinary applicability, combined with the timeliness of AI-driven scientific discovery and test-time compute scaling, gives Paper 1 a substantially wider potential impact on the broader scientific community beyond just AI and machine learning research.
Paper 2 likely has higher impact due to its broad, general-purpose framework (SimpleTES) for scaling evaluation-driven discovery across many scientific domains, with strong empirical results on 21 problems and concrete new records (algorithm speedups, quantum routing improvements, combinatorial constructions). Its methodological contribution (principled scaling dimensions, trajectory supervision, post-training with generalization) is broadly applicable beyond any single field and is timely for LLM-based scientific workflows. Paper 1 is innovative for drug design but is narrower in scope and more dependent on proxy evaluations/structure-model biases, potentially limiting near-term real-world translation.
Paper 1 has higher scientific impact due to its broad cross-domain applicability and contribution to fundamental science. By formalizing test-time scaling for LLM-driven scientific discovery, it advances multiple distinct fields simultaneously—yielding novel mathematical theorems, faster optimization algorithms, and better quantum routing. While Paper 2 offers significant engineering improvements for edge LLM inference, Paper 1 accelerates the fundamental scientific method itself. Its novelty in leveraging test-time compute and proven ability to surpass best-known human results guarantee a wider, more profound impact across the broader scientific community.
Paper 2 addresses a highly timely and broadly impactful problem—scaling LLM-driven scientific discovery—and demonstrates concrete, verifiable breakthroughs across six domains (LASSO 2x speedup, quantum circuit routing, new Erdos constructions). Its framework has broad cross-disciplinary applications and generalization potential. Paper 1 is rigorous and offers valuable theoretical insight into low-rank optimizer non-identifiability, but its impact is narrower, confined to memory-efficient training methods. Paper 2's demonstrated real-world discoveries and general applicability give it substantially higher potential impact.
Broad, open, reproducible demonstration that simple structured scaling with a small open model matches or beats complex proprietary-model discovery systems, but many gains are marginal and the algorithmic novelty is modest.