Simulate, Reason, Decide: Scientific Reasoning with LLMs for Simulation-Driven Decision Making
Yuhan Yang, Ruipu Li, Alexander Rodríguez
Abstract
Scientific simulators are increasingly being integrated into LLM-driven systems for high-stakes simulation-driven decision-making. However, existing frameworks primarily use LLMs to generate, calibrate, or execute simulators, treating them as black-box interfaces rather than as structured mechanistic systems that can be reasoned about. As a result, current approaches lack the ability to identify, represent, and reason about the assumptions and mechanisms underlying simulator behavior, limiting transparency, auditability, and decision justification. We introduce MechSim, a mechanism-grounded neuro-symbolic reasoning framework for executable scientific simulators. Unlike prior neuro-symbolic approaches that primarily reason over static symbolic structures, MechSim enables LLM agents to reason about the mechanisms, assumptions, and execution behavior of scientific simulators. Our framework represents simulators through a shared structured schema capturing assumptions, variables, mechanism dependencies, and execution traces. On top of this representation, LLM agents operate as constrained reasoning engines that generate structured, evidence-grounded explanations linking simulator outcomes to their underlying mechanisms. We evaluate our approach across multiple high-stakes domains and show that it improves mechanism-level explanation quality, simulator analysis, and downstream decision-making reliability.
AI Impact Assessments
(1 models)Scientific Impact Assessment: MechSim — Mechanism-Grounded Neuro-Symbolic Reasoning for Simulation-Driven Decision Making
1. Core Contribution
MechSim addresses a genuine gap at the intersection of LLM-based reasoning and scientific simulation: while LLMs are increasingly used to generate, calibrate, or execute simulators, they lack structured ways to reason *about* simulator internals — assumptions, mechanisms, and how those drive outputs. The paper's core novelty is a neuro-symbolic framework that constructs "mechanism graphs" from simulator definitions and execution traces, then constrains LLM reasoning to traverse valid mechanistic pathways when generating explanations and recommendations. The framework decomposes into four components: contextual grounding, mechanism-level grounding (including graph construction, calibration via ABC, and sensitivity analysis), constrained mechanism-level reasoning (producing structured explanations E = (I, P, Z, C)), and verification checks. This is instantiated as a multi-agent system with specialized LLM agents.
The key intellectual contribution is shifting LLM-simulator interaction from "LLM calls simulator and summarizes output" to "LLM reasons over simulator structure, traces mechanistic propagation paths, and grounds claims in both execution behavior and retrieved scientific evidence." This is a meaningful conceptual advance.
2. Methodological Rigor
Strengths in experimental design:
Concerns:
3. Potential Impact
Practical applications: The framework addresses a real need in high-stakes domains where simulation-driven decisions require justification and auditability. Public health officials, supply chain managers, and policymakers would benefit from explanations that trace recommendations back to specific simulator mechanisms and assumptions.
Broader influence: MechSim could influence several adjacent areas:
Limitations on impact: The framework requires substantial domain-specific setup — simulator definitions must be parseable, calibration pipelines must be configured, and domain corpora must be available. Scalability to truly complex simulators (e.g., climate GCMs with millions of state variables) remains unaddressed.
4. Timeliness & Relevance
The paper is highly timely. LLM agents are rapidly being deployed for scientific workflows, and the gap between "LLM can call a simulator" and "LLM can reason about why a simulator produces certain outputs" is becoming increasingly important. The COVID-19 pandemic vividly demonstrated the need for transparent, mechanism-grounded reasoning about competing epidemic models. The integration of scientific evidence retrieval (inspired by OpenScholar) with mechanistic reasoning is well-motivated by current trends in retrieval-augmented generation.
5. Strengths & Limitations
Key Strengths:
Notable Weaknesses:
6. Additional Observations
The paper's appendix is thorough and includes complete prompt templates, which significantly aids reproducibility. The example explanation output (Appendix C.2) demonstrates genuinely useful mechanistic reasoning that would be valuable for decision-makers. However, the paper would benefit from a user study with domain experts to validate whether the explanations actually improve human decision-making quality.
Generated Jun 5, 2026
Comparison History (17)
Paper 1 presents a novel theoretical framework combining online learning, Pandora's Box theory, and LLM cascading with rigorous regret bounds. It introduces a new feedback structure (output-mediated), develops GMM-based estimation for reservation indices, and proves √T regret guarantees. This provides foundational algorithmic contributions applicable beyond LLMs to broader sequential decision-making. Paper 2 introduces a useful engineering framework (MechSim) for simulator reasoning but is more incremental, combining existing ideas (neuro-symbolic reasoning, structured schemas) without comparable theoretical depth or generalizable methodological innovation.
AutoLab introduces a novel benchmark addressing a critical gap in evaluating frontier AI models on long-horizon iterative tasks, a fundamental capability for autonomous scientific research. With 36 expert-curated tasks, evaluation of 17 SOTA models, and fully open-sourced artifacts, it provides high-utility infrastructure for the rapidly growing AI agents community. Its finding that persistence and iterative refinement matter more than initial solution quality is an important insight. While MechSim addresses an interesting niche in simulator reasoning, AutoLab has broader applicability, stronger methodological rigor with comprehensive model comparisons, and higher timeliness given the surge in AI agent development.
Paper 1 targets an immediate, widely encountered bottleneck in personal AI agents: trustworthy long-term memory retrieval as a security/control boundary. It introduces a lightweight, deployable gating module that plugs into existing memory stacks without LLM modification, and evaluates across multiple mainstream frameworks and real-world agent settings—suggesting strong methodological rigor and high near-term adoption potential. Its impact spans security, alignment, retrieval, and agentic systems, and directly addresses timely failure modes (jailbreaks, sycophancy, tool drift). Paper 2 is promising but likely narrower and harder to standardize across diverse simulators.
Paper 2 offers broader interdisciplinary impact by bridging LLMs with scientific simulators for high-stakes decision-making. While Paper 1 provides a strong, systematized framework for knowledge infusion in generative AI, Paper 2 tackles the fundamental challenge of transparency and auditability in AI-driven scientific reasoning. By enabling LLMs to understand and reason about the internal mechanisms of simulators rather than treating them as black boxes, Paper 2 has the potential to accelerate robust scientific discovery and application across numerous STEM fields.
Agents' Last Exam (ALE) has higher potential impact due to its breadth (1K+ tasks across 55 subfields, 13 industry clusters), large-scale expert collaboration (250+ experts), and its role as a living benchmark addressing a fundamental gap between AI benchmark performance and real-world economic deployment. It establishes infrastructure that will shape how the entire AI agent community measures progress. While MechSim introduces a valuable neuro-symbolic framework for simulator reasoning, its scope is narrower. ALE's potential to redirect research priorities across the AI field toward economically meaningful tasks gives it broader and more lasting impact.
Paper 2 has higher potential scientific impact due to broader real-world applicability and cross-domain relevance: mechanism-grounded reasoning over executable simulators directly targets high-stakes decision-making, transparency, and auditability across many sciences and engineering fields. Its neuro-symbolic schema and trace-based reasoning could generalize to diverse simulators and governance needs, making it timely amid increasing simulator+LLM deployment. Paper 1 is highly novel and rigorously benchmarked with striking theorem-proving results, but its impact is more concentrated within formal methods/automated reasoning and depends on continued scaling of LLM-based provers.
Paper 2 likely has higher impact due to timeliness and breadth: scalable long-horizon agent memory is a near-term bottleneck across most deployed agent applications, and a first systems-level characterization with taxonomy, profiling harness, comparative evaluation across many systems, and actionable recommendations can directly influence both research and production stacks. Its methodological rigor (measurement framework + multi-system benchmarking) and broad applicability across domains and infrastructure layers suggest wider adoption. Paper 1 is novel and valuable for simulator-grounded transparency, but its impact may be narrower to simulation-driven decision workflows and depends on adoption of its schema and neuro-symbolic pipeline.
Paper 1 addresses a critical challenge in AI: enabling LLMs to reason transparently about scientific simulators for high-stakes decision-making. Its mechanism-grounded framework has broad, cross-disciplinary applications in any field relying on computational modeling. In contrast, Paper 2 focuses on instilling virtuous behavior in MARL agents within a specific board game environment. While relevant to AI ethics, Paper 1's generalizable approach to scientific reasoning offers greater potential for immediate, real-world impact across diverse scientific domains.
Paper 1 addresses a fundamental and widely recognized problem in LLM-based dialogue systems—distribution shift in multi-turn RL—with both theoretical analysis (quadratic compounding) and a practical framework (Calibrated Interactive RL). The clear decomposition of distribution shift sources and the unified solution has broad applicability across dialogue and RL communities. Paper 2 introduces an interesting neuro-symbolic framework for simulator reasoning, but targets a narrower niche. Paper 1's theoretical contributions combined with empirical validation and its relevance to the rapidly growing RLHF/dialogue agent field give it higher potential impact.
Paper 2 establishes a fundamental theoretical impossibility result (a quadrilemma) about AI explainability that has broad implications across all of AI, including governance and regulation. Such foundational impossibility theorems (akin to No Free Lunch or Arrow's theorem) tend to have lasting, wide-reaching impact across multiple fields. Paper 1, while practical and valuable, presents an incremental framework for a specific application domain. Paper 2's mathematical proof constrains what is achievable in AI explainability universally, making it more likely to be widely cited and influential in both theory and policy.
Paper 1 introduces a novel, generalizable neuro-symbolic framework (MechSim) that addresses a critical gap in AI for Science by making black-box simulators interpretable and auditable. Its potential for high-stakes, real-world applications across multiple scientific domains promises a broader and more enduring methodological impact. While Paper 2 offers timely and important insights into AI ethics and persuasion, its contribution is primarily observational and tied to a specific, discontinued experiment, limiting its broader technical applicability compared to Paper 1.
Paper 2 has higher likely impact due to its broad relevance to current LRM/chain-of-thought research, offering a clear evaluation protocol (prefix-level trajectory/sufficiency) and empirically demonstrating accuracy gains (up to 21%) plus a new failure mode (harmful overthinking) affecting multimodal and language tasks. It is timely for test-time compute scaling and reliability, with immediate applications to decoding, early-exit methods, and safety. Paper 1 is novel and valuable for simulator-centered decision systems, but its impact is narrower to domains using scientific simulators and depends more on integration complexity and availability of executable simulators.
Paper 2 (MechSim) addresses a more impactful problem—integrating scientific simulators with LLM reasoning for high-stakes decision-making—with a novel neuro-symbolic framework that has direct real-world applications across multiple domains. While Paper 1 (FalsifyBench) makes a solid contribution by benchmarking inductive reasoning in LLMs with insights about negative testing, it is primarily a diagnostic evaluation benchmark. Paper 2 introduces a new framework (MechSim) with broader applicability to scientific simulation, transparency, and auditability, addressing critical needs in AI-assisted decision-making that span multiple high-stakes fields.
Paper 1 is likely to have higher scientific impact due to greater novelty and broader cross-domain applicability: it introduces a mechanism-grounded neuro-symbolic framework that makes simulators auditable and explainable, addressing transparency and justification in high-stakes simulation-driven decisions across multiple scientific domains. This can influence both AI reasoning and computational science workflows. Paper 2 offers a timely, practical reframing of hallucination detection via OOD geometry with training-free detectors, but the conceptual leap is more incremental and narrower (primarily LLM safety/evaluation) and may face robustness limits across settings.
Paper 1 presents a foundational, domain-agnostic framework (MechSim) for LLM reasoning over scientific simulators. This addresses a critical bottleneck in AI for science by moving beyond black-box execution to enable mechanism-grounded neuro-symbolic reasoning. Its potential impact spans across multiple high-stakes scientific disciplines, making its breadth and methodological novelty significantly higher than Paper 2, which, while highly rigorous and practically valuable for healthcare, is fundamentally constrained to tabular clinical risk prediction.
Paper 2 has higher potential impact due to broader, timelier applicability: mechanism-grounded reasoning over scientific simulators targets high-stakes decision-making across many domains (engineering, climate, epidemiology, policy) and directly addresses transparency/auditability—key current concerns for AI deployment. Its schema for assumptions, dependencies, and execution traces plus constrained, evidence-grounded explanations suggests stronger methodological rigor and a clearer path to real-world adoption than KG QA gains. Paper 1 is novel and effective within KG question answering, but the scope and cross-field impact are narrower.
Paper 2 introduces a novel neuro-symbolic framework (MechSim) that addresses a fundamental gap in how LLMs interact with scientific simulators across multiple high-stakes domains. Its breadth of impact spans scientific computing, AI reasoning, and decision-making, with clear real-world applications in domains requiring transparency and auditability. Paper 1, while addressing an interesting and practical problem (handoff debt in coding agents), is more narrowly focused on software engineering workflows and coding agent benchmarks. Paper 2's contribution to mechanistic reasoning and its cross-domain applicability give it broader and deeper potential scientific impact.