WebGameBench: Requirement-to-Application Evaluation for Coding Agents via Browser-Native Games

Paper 2 likely has higher scientific impact: it targets a core, timely limitation of foundation agents—test-time adaptation for ad-hoc multi-agent coordination—using a large-scale, rigorously controlled benchmark (ICRL4AHT) with diverse teammate distributions, clear train/test shifts, and high-throughput JAX Overcooked-V2 evaluation over millions of transitions. Its negative results (baselines underperforming random) are broadly informative and can redirect algorithm development across RL, multi-agent systems, and LLM-agent research. Paper 1 is novel and useful for coding-agent evaluation, but its domain (browser games) is narrower and impact is more application-benchmark specific.

Paper 2 introduces a novel method (VISION) that bridges in-context learning from LLMs with graph few-shot learning, addressing fundamental limitations in the field. It has broader methodological impact across graph learning, meta-learning, and in-context learning communities. While Paper 1 (WebGameBench) is a solid benchmark contribution for evaluating coding agents, benchmarks typically have narrower impact than novel methods. Paper 2's approach of unifying unsupervised meta-learning with graph in-context learning offers more transferable insights and broader applicability across multiple domains.

Paper 1 provides a profound, rigorous mathematical foundation for deep learning architectures using lattice theory and mathematical morphology. By explaining the algebraic reasons behind the representational power of CNNs and characterizing new idempotent layers, it offers lasting theoretical insights that could fundamentally influence future network designs. Paper 2 is a timely and useful empirical benchmark for coding agents, but its long-term scientific impact is likely narrower and shorter-lived compared to the foundational theoretical contributions of Paper 1.

Paper 1 tackles a critical safety bottleneck in the physical deployment of AI (robotics) by reducing Vision-Language Model hallucinations. Its novel pseudocode-guided reasoning framework achieves SOTA results, directly addressing real-world reliability. While Paper 2 presents a valuable evaluation benchmark for coding agents, Paper 1's methodological innovation and direct application to physical automation offer a broader and more critical scientific impact.

Paper 1 offers a novel theoretical formalization for a critical bottleneck in agentic AI: trust calibration and progressive autonomy. By mapping this human-in-the-loop problem to Preferential Bayesian Optimization, it provides a rigorous, mathematically grounded framework with broad applicability across any domain requiring human oversight of AI. While Paper 2 presents a useful and timely empirical benchmark for coding agents, Paper 1's methodological innovation and potential to fundamentally shape how we design safe, semi-autonomous AI systems give it a higher potential for foundational scientific impact.

Paper 1 challenges a fundamental assumption in AI—that training on code inherently improves general reasoning. By using a massive 10T-token corpus to isolate the true driver of reasoning (structured reasoning traces rather than pure code), it provides critical insights for pretraining data curation. This impacts the development of all future large language models, giving it exceptional breadth and timeliness. While Paper 2 offers a valuable benchmarking tool for coding agents, Paper 1's findings address the core architecture of model cognition, giving it a much broader and deeper scientific footprint.

Paper 1 addresses a more impactful domain—LLM-driven engineering design—with broader real-world applications spanning manufacturing, HPC, and simulation. Its multi-agent framework with systematic benchmarking across workflow complexity, RAG contributions, and HPC orchestration provides deeper methodological contributions. The engineering design focus has higher potential for cross-disciplinary impact (mechanical engineering, materials science, HPC). Paper 2, while well-constructed, evaluates coding agents on browser games—a narrower domain with less transformative potential. Paper 1's findings on model scaling, conditional reasoning failures, and multi-step instruction degradation offer more generalizable scientific insights.

Paper 2 likely has higher scientific impact due to strong timeliness and broad applicability: it provides a scalable, end-to-end benchmark for coding agents focused on delivered application behavior, with real-browser runtime evaluation and partial human validation. This can directly influence how agentic coding systems are trained, compared, and deployed across academia and industry, affecting multiple subfields (LLM agents, software engineering, HCI, evaluation). Paper 1 is technically rigorous and novel in theory (variance-aware minimax regret for MNL MDPs) but its impact is narrower and more incremental within RL theory.

CBT-Audio addresses a significant gap at the intersection of AI, mental health, and audio language models. It introduces a novel dataset for an underexplored modality (audio) in CBT evaluation, with clear clinical relevance. The finding that audio provides complementary information to text—especially when verbal content and vocal delivery diverge—has meaningful implications for therapeutic AI systems. WebGameBench, while well-constructed, addresses a narrower problem in coding agent evaluation with less cross-disciplinary impact. CBT-Audio's contributions to mental health AI, multimodal evaluation, and clinical applications give it broader and more lasting scientific impact.

WebGameBench addresses a more fundamental gap in coding agent evaluation by shifting focus from code-level metrics to delivered application quality—a paradigm shift with broad implications for how we assess AI coding systems. It introduces a concrete, reproducible benchmark with 111 tasks, 12 agents, and human-validated evaluation, filling a clear void in the field. MetaCogAgent, while novel in integrating metacognition into multi-agent LLMs, builds incrementally on existing multi-agent frameworks and relies on a self-constructed benchmark (MetaCog-Eval) that may limit external adoption. WebGameBench's practical grounding and applicability to the rapidly growing coding agent ecosystem give it broader and more timely impact.