DecisionBench: A Benchmark for Emergent Delegation in Long-Horizon Agentic Workflows

Paper 1 introduces a comprehensive benchmark with extensive empirical evaluation (23,375 task instances), quantitative findings on delegation behavior, and releases reusable artifacts enabling reproducible research on multi-agent orchestration. Its findings about routing fidelity and counterfactual delegation ceilings provide concrete, actionable insights for the community. Paper 2 presents an interesting architectural concept (event-sourced agent runtime) but is primarily a design paper that explicitly states it does not demonstrate its claimed benefits, limiting its immediate empirical impact. Benchmarks tend to have outsized community impact by enabling standardized comparison.

While Paper 1 provides a valuable infrastructure benchmark for agentic workflows, Paper 2 tackles a fundamental scientific question regarding how multimodal models reason about human attributes. By exposing the 'Prejudice Gap' and shifting evaluation from superficial score prediction to grounded behavioral reasoning, it addresses critical issues in AI safety, fairness, and interpretability, promising broader interdisciplinary impact across AI, psychology, and HCI.

Paper 1 addresses a fundamental gap in MLLM evaluation—whether models truly understand personality or rely on superficial cues—introducing a novel task (GPR), a comprehensive dataset (MM-OCEAN), and revealing the striking 'Prejudice Gap' across 27 models. This exposes a deep issue (right answer, wrong reasoning) relevant to AI safety, trustworthiness, and social cognition broadly. Paper 2 contributes a useful benchmark for agentic delegation but addresses a narrower, more engineering-focused problem with less surprising findings. Paper 1's conceptual contribution and implications for responsible AI deployment give it broader and more lasting impact.

Paper 2 introduces a comprehensive benchmark for multi-agent delegation, an emerging and critical area in LLM research. Benchmarks typically drive significant scientific progress by establishing standard evaluation metrics, highlighting performance gaps, and enabling comparisons across future methods. While Paper 1 offers a highly practical and novel approach to GUI automation efficiency, Paper 2's potential to become a foundational testing substrate for orchestration and long-horizon agentic workflows gives it a broader and longer-lasting potential scientific impact.

DecisionBench addresses a timely and broadly impactful problem—emergent delegation in multi-agent LLM workflows—with a rigorous benchmark methodology, large-scale experiments (23K+ instances), and clear findings that expose significant unrealized headroom for future orchestration methods. It provides a reusable evaluation substrate for the rapidly growing agentic AI community. Paper 2 (LBW-Guard), while showing practical training stability improvements, addresses a narrower systems-level concern with limited model/dataset scope (WikiText-103, LoRA fine-tuning) and incremental contribution over existing training stability techniques. DecisionBench's broader community utility and timeliness give it higher impact potential.

Paper 1 makes a fundamental theoretical contribution to reinforcement learning by establishing minimax optimal variance-aware regret bounds for MNL mixture MDPs, with both upper and lower bounds that fully characterize the regret complexity. This resolves an open theoretical question with rigorous proofs and has broad implications across RL theory. Paper 2 introduces a useful benchmark for LLM delegation workflows, but benchmarks tend to have more transient impact, and its key finding—that quality is indistinguishable across conditions—limits its immediate practical influence. Paper 1's theoretical results are more enduring and foundational.

Paper 1 reveals a highly counterintuitive phenomenon in embodied AI—that higher observation fidelity and perfect information can actually degrade LLM problem-solving performance. This challenges prevailing assumptions and evaluation methodologies in the field, likely sparking significant follow-up research into LLM reasoning failures and perceptual interactions. While Paper 2 provides a valuable and rigorous benchmark for agentic workflows, Paper 1 offers a more fundamental conceptual shift with broad implications for robotics and AI safety.

Paper 2 likely has higher scientific impact due to a more novel, generalizable methodological contribution: an automated benchmark-generation pipeline coupled with a formal verification guarantee (cycle-consistency ensuring uniqueness), reducing annotation cost while controlling hallucinations. This can scale to many task families and offers a principled way to create robust reasoning benchmarks, with relevance beyond LLMs (program synthesis, formal methods, cognitive evaluation). Paper 1 is valuable infrastructure for delegation/orchestration evaluation, but is narrower in scope and more incremental as a benchmark substrate rather than a new formalizable paradigm.

AutoRubric-T2I addresses a widely impactful problem—aligning text-to-image generation with human preferences—with a novel, practical framework that dramatically reduces data requirements (0.01% of annotated data) while improving interpretability and performance. It demonstrates strong results on multiple benchmarks and downstream RL tasks. DecisionBench introduces a useful evaluation substrate for multi-agent delegation but reports largely negative findings (quality indistinguishable across conditions) and is more niche in scope. Paper 2's combination of methodological novelty, practical efficiency gains, and broad applicability to the rapidly growing T2I field gives it higher potential impact.

Paper 1 has higher likely impact: it introduces a reusable, large-scale benchmark substrate for emergent delegation/orchestration in long-horizon agent workflows with standardized interfaces, metrics, deterministic annotations, and extensive reference sweeps plus released artifacts—directly enabling rigorous, comparable progress across many LLM-agent methods and vendors. Its applications (agent routing, tool/model selection, cost/latency-quality tradeoffs) are immediate and broadly relevant to ML systems and deployment. Paper 2 is timely and methodologically solid observational science with cross-disciplinary interest, but its contributions are more domain-specific and less likely to catalyze widespread methodological advances.

Paper 1 introduces a comprehensive benchmark for an emerging and critical area (agentic delegation), which often dictates the trajectory of future research. By exposing a significant 15-31% unrealized performance headroom, it sets a clear target for the community, likely driving high citation volume and broader structural impact compared to the specific algorithmic optimization proposed in Paper 2.

Paper 2 has higher likely impact because it delivers a concrete, reproducible benchmark substrate with released data, metrics, and large-scale empirical characterization (n=23,375), enabling immediate methodological comparisons and progress on delegation/orchestration—an increasingly timely real-world need for agentic systems. Its multi-axis metrics and counterfactual ceiling provide rigorous evaluation signals beyond end-task quality, with applicability across LLM routing, systems, and HCI. Paper 1 is a compelling, novel framing and taxonomy, but as a position paper it is less methodologically grounded and offers fewer directly actionable artifacts for the community.

Paper 1 advances fundamental understanding of Multimodal Large Language Models through mechanistic interpretability, pinpointing the exact internal dynamics causing modality-conflict hallucinations. Its proposed causal intervention directly addresses a critical AI safety and reliability issue without requiring retraining. While Paper 2 offers a valuable benchmark for multi-agent systems, Paper 1 provides deeper methodological innovation and broader immediate impact by uncovering the architectural root causes of hallucinations and offering a scientifically grounded, inference-time solution.

Paper 2 likely has higher scientific impact because it introduces a broadly reusable benchmark substrate (tasks, peer-model pool, interface, annotations, metrics, and large-scale runs) that can standardize evaluation of delegation/orchestration across many future methods and vendors—high breadth, timeliness, and real-world relevance. Its methodological rigor is supported by multi-axis metrics, controlled conditions, and large n. Paper 1 is a solid algorithmic contribution with measurable gains, but its impact is narrower (on-policy RL for reasoning/optimization) and may be superseded by alternative RL objectives, while DecisionBench can become shared infrastructure for the field.

Paper 2 likely has higher impact: it introduces a broadly useful, standardized benchmark and evaluation substrate for emergent delegation/orchestration across many models and vendors, with large-scale empirical characterization and released artifacts. This directly enables reproducible, comparable progress in a timely area (agentic workflows, routing, cost/latency tradeoffs) and can influence multiple fields (LLM systems, HCI, benchmarking, applied ML). Paper 1 is novel for MARL communication using LLMs but is narrower in scope/application and less likely to become a shared community infrastructure.

GRAM introduces a fundamentally new framework for neural reasoning by combining recursive latent-state refinement with probabilistic multi-trajectory computation, addressing core limitations of deterministic recursive reasoning models. This has broad theoretical implications for how neural systems perform extended computation, with applications spanning structured reasoning, constraint satisfaction, and generation. Paper 2, while a solid engineering contribution providing a useful benchmark for agentic delegation, is more narrowly scoped to evaluating LLM orchestration workflows. GRAM's methodological novelty (variational inference for recursive reasoning, inference-time scaling via depth and sampling) is likely to inspire more follow-on research across multiple subfields.

Paper 1 presents a highly novel algorithmic approach combining Bayesian optimization and LLM-driven dynamic feature elicitation for prompt tuning. This addresses a critical, ubiquitous problem in AI deployment—optimizing system prompts with only aggregate black-box feedback. Its innovative methodology offers broad implications for optimizing discrete natural language artifacts, likely yielding higher immediate and cross-disciplinary scientific impact than the benchmark proposed in Paper 2, despite Paper 2's rigorous evaluation of agentic delegation.

Paper 2 addresses a critical and timely AI safety concern—jailbreaking Large Reasoning Models—with a novel mechanistic insight linking attention patterns to attack success, plus a concrete RL-based method. This has broader immediate impact across AI safety, alignment, and model deployment. The attention-guided reward mechanism is innovative and actionable for both offensive and defensive research. Paper 1, while thorough as a benchmarking substrate for agentic delegation, addresses a narrower community and its key finding—that quality is indistinguishable across conditions—limits its immediate transformative impact.

Paper 1 introduces a novel theoretical framework bridging behavioral economics (prospect theory) with strategic classification in ML, addressing a fundamental limitation in existing models. This interdisciplinary contribution has broad implications for deploying ML systems in real-world settings where human behavior deviates from rationality. Paper 2, while useful as a benchmark for LLM-agent delegation, is more incremental—benchmarks have shorter lifespans and narrower conceptual contributions. Paper 1's formalization of behaviorally realistic strategic agents opens a new research direction with stronger long-term theoretical and practical impact.

Paper 1 has higher impact potential due to a broader, more reusable benchmark substrate for emergent delegation/orchestration across long-horizon agentic workflows, spanning multiple task suites and many models/vendors with rigorous multi-axis metrics and large-scale evaluation. It introduces actionable diagnostics (routing fidelity, vendor self-preference, counterfactual ceiling) that directly enable progress and comparisons across methods and labs. Paper 2 targets an important AV planning niche and offers useful negative results plus a benchmark, but its scope is narrower and evidence of measurable gains is limited.