Responsible Agentic AI Requires Explicit Provenance
Jinwei Hu, Xinmiao Huang, Qisong He, Youcheng Sun, Yi Dong, Xiaowei Huang
Abstract
Agentic AI is rapidly proliferating across diverse real-world domains such as software engineering, yet public trust has not kept pace. The central reason is that responsibility, despite being widely discussed, remains a subjective and unenforced concept, as no current agentic framework produces the quantifiable, traceable, and interventionable provenance needed to assign it when harm emerges from compositions no single party designed. We position that what is missing is not better benchmark-level evaluation but across the full agentic lifecycle, which is the only viable basis for making responsibility computable and actionable. We advance this agenda along four axes: establishing such provenance is a structural necessity by identifying responsibility gaps across sociotechnical dimensions, formalizing it must encode through a causal attribution function and responsibility tensor, discussing it can be made computable across four lifecycle layers with preliminary experiments showing that provenance is estimable and interveneable online before irreversible harm accumulates, and examining bears responsibility through a concrete agentic incident. Explicit provenance is not a discretionary refinement but the necessary condition for responsible agentic AI, and no stakeholder across its ecosystem can afford to treat it as optional.
AI Impact Assessments
(1 models)Scientific Impact Assessment
Core Contribution
This paper positions "explicit provenance" as the necessary infrastructure for making responsibility computable and actionable in agentic AI systems. The authors argue that current trustworthy AI approaches, focused on per-component benchmarking and auditing, are structurally insufficient for agentic systems where harm emerges from compositional, multi-step trajectories involving multiple stakeholders. The contribution is organized along four axes: why provenance is necessary (sociotechnical responsibility gaps), what it must encode (a causal attribution function and responsibility tensor), how it can be made computable (a four-layer lifecycle framework with preliminary neuro-symbolic experiments), and who bears responsibility (illustrated through a concrete incident).
The main novelty lies in the formalization attempt—particularly the responsibility tensor R ∈ [0,1]^{|P|×|Ω|×|D|} and the causal contribution function κ(p, ω, τ)—which tries to bridge the gap between philosophical responsibility discourse and computational operationalization. The four-layer lifecycle framework (Design, Engineering, Deployment, Experience) provides a structured research agenda.
Methodological Rigor
The paper operates primarily as a position paper with preliminary empirical support. The formalizations, while conceptually motivated, raise significant concerns:
The causal contribution function κ(p, ω, τ) = Pr[ω|τ] − Pr[ω|τ_{−p}] relies on counterfactual trajectories that are acknowledged to be theoretical anchors rather than computable quantities. The paper does not adequately address the fundamental problem that counterfactual reasoning in complex multi-agent systems is computationally intractable in general. The gap between the formal definition and what the experiments actually measure (AUPRC of failure prediction from execution prefixes) is substantial—predicting failure from prefixes is not the same as computing counterfactual causal contributions.
The responsibility tensor is formally defined but its practical instantiation remains largely illustrative. The dimension weights w_k are acknowledged as requiring human judgment, which somewhat undermines the "computable" framing. The completeness condition (responsibilities sum to 1) is a normative choice presented as a formal requirement without sufficient justification for why responsibility should be zero-sum.
The preliminary experiments (Section 5.5) demonstrate that neuro-symbolic monitors can predict trajectory failure from execution prefixes across four benchmarks, with AUPRC substantially above random baselines. This is a meaningful empirical contribution, but the connection to the broader provenance framework is indirect. Detecting that something is going wrong is not equivalent to attributing causal responsibility to specific parties—a significant conceptual leap that the paper acknowledges but does not bridge.
The Example 1 (WebArena responsibility assignment) is illustrative rather than validated. The mapping from DFA states to responsibility assignments involves substantial interpretive judgment that the paper presents as more mechanical than it actually is.
Potential Impact
The paper addresses a genuinely important problem at the intersection of AI safety, governance, and deployment. Several aspects could influence the field:
1. Framing contribution: The articulation that responsibility requires three simultaneous properties (quantifiability, traceability, interventionability) provides a useful analytical framework for the responsible AI community.
2. Research agenda: The four-layer lifecycle framework (L1-L4) identifies concrete research directions that could organize future work, particularly around compositional verification and population-scale monitoring.
3. Bridge-building: The paper attempts to connect technical AI safety with legal, ethical, and regulatory frameworks, which is valuable given the current policy moment around AI governance (EU AI Act, AI Liability Directive).
4. Practical monitoring: The neuro-symbolic monitoring approach, while preliminary, suggests a viable path toward runtime provenance that could be adopted in production agentic systems.
However, the impact is tempered by the significant gap between the formal framework and its practical realizability. The paper risks setting up an impossibly high standard (full counterfactual causal attribution) that may discourage rather than enable practical progress.
Timeliness & Relevance
The paper is highly timely. Agentic AI deployment is accelerating rapidly (as evidenced by the industry surveys cited), and the governance gap is real. The EU AI Liability Directive's application to agentic systems is an active policy question, and frameworks for attributing responsibility in compositional AI systems are urgently needed. The paper correctly identifies that this is a structural problem rather than a matter of incremental improvement to existing evaluation paradigms.
Strengths
Limitations
Overall Assessment
This is a well-motivated position paper that identifies a genuine and important problem in agentic AI governance. Its primary contribution is conceptual framing rather than technical innovation. The formal framework is ambitious but the gap between formalism and empirical validation is significant. The paper would benefit from a more honest assessment of what is currently achievable versus aspirational, and from deeper engagement with the computational complexity of counterfactual reasoning in multi-agent systems.
Generated May 19, 2026
Comparison History (22)
GeoX presents a novel, concrete technical framework combining self-play reinforcement learning with executable program verification for geospatial reasoning—a growing field with significant real-world applications (remote sensing, urban planning, disaster response). It demonstrates measurable improvements (5.5 points) over strong baselines without requiring large-scale human annotation, and releases a benchmark. Paper 2, while addressing an important topic (responsible AI provenance), is primarily a position/framework paper with only preliminary experiments. Its impact depends on adoption of proposed frameworks, which historically faces challenges. GeoX's methodological contribution and empirical results offer more immediate and citable scientific impact.
Paper 1 introduces a concrete algorithmic improvement (DMPO) to solve a critical bottleneck (mode collapse) in highly relevant RL methods like GRPO. Its strong empirical validation across diverse reasoning tasks suggests immediate and widespread utility in the rapidly advancing field of LLM reasoning. While Paper 2 addresses vital AI safety concerns, its position-paper nature and conceptual focus will likely yield less direct, measurable scientific impact and fewer follow-up algorithmic innovations compared to Paper 1.
Paper 2 offers higher immediate scientific impact due to its strong methodological rigor and concrete solutions to a critical bottleneck in modern AI: LLM training instability and compute waste. While Paper 1 addresses an important conceptual gap in AI governance, it is primarily a position paper with preliminary experiments. In contrast, Paper 2 provides a novel algorithmic intervention (LBW-Guard) supported by extensive empirical validation across multiple LLM scales (up to 14B). Its ability to maintain training stability under extreme stress has massive, immediate real-world applications for reducing the exorbitant costs of foundation model training.
Paper 1 addresses a universal and critical bottleneck in the deployment of agentic AI systems across all domains. By introducing a formal framework for explicit provenance and a computable responsibility tensor, it offers a foundational, cross-disciplinary solution that spans software engineering, AI ethics, and law. While Paper 2 provides highly valuable empirical safety data for autonomous driving, Paper 1's conceptual framework has broader potential to shape the fundamental architecture and regulation of future multi-agent AI ecosystems.
Paper 1 presents a practical, validated methodology (Noise2Noise denoising for Raman spectroscopy) with concrete experimental results demonstrating significant speedup (5ms acquisitions) while maintaining spectral fidelity. It addresses a real bottleneck in high-throughput spectroscopy with a transferable framework. Paper 2 is a position/framework paper on AI provenance that, while timely, lacks substantial empirical validation and operates in a crowded responsible-AI discourse space. Paper 1's concrete methodological contribution and reproducible pipeline are more likely to generate direct citations and adoption across spectroscopic communities.
Paper 1 addresses a fundamental and increasingly critical problem—responsibility and provenance in agentic AI—with novel formal contributions (causal attribution function, responsibility tensor) that have broad implications across AI safety, governance, policy, and multiple application domains. Its framework-agnostic, structural approach to a widely recognized trust gap gives it high potential for cross-disciplinary impact and policy relevance. Paper 2, while practically useful, presents an incremental engineering contribution analyzing specific interaction paradigms within a single framework (buddyMe), with narrower scope and limited generalizability beyond its specific system context.
Paper 2 tackles a critical, overarching challenge in AI—accountability and safety in agentic systems—by proposing a formal framework for explicit provenance. This addresses urgent sociotechnical and ethical needs, offering broad, cross-disciplinary impact spanning AI, policy, and engineering. While Paper 1 offers a valuable benchmark for evaluating specific spatial and temporal reasoning capabilities in LLMs, its scope and applications are significantly narrower than the foundational safety and responsibility framework proposed in Paper 2.
Paper 1 demonstrates higher scientific impact through its concrete, large-scale empirical validation (3,000 trials, 119 knowledge infrastructures across 14 Earth-science domains) and addresses a critical practical barrier—democratizing access to process-based simulation models. It delivers a generalizable toolkit (KDT) with measurable performance gains (84% vs 40% success). Paper 2, while addressing an important problem (provenance for responsible AI), is primarily a position/framework paper with only preliminary experiments. Paper 1's combination of methodological rigor, immediate practical utility, and breadth across Earth sciences gives it stronger near-term and long-term impact potential.
Paper 1 targets a cross-cutting, timely bottleneck for deploying agentic AI safely: making responsibility computable via explicit, lifecycle-wide provenance. If adopted, its concepts (causal attribution, responsibility tensor, online interveneable provenance) could influence standards, auditing, governance, and system design across many application domains, yielding broad, long-lived impact. Paper 2 is technically strong and practically valuable for MILP, but its impact is narrower to optimization/solver communities and may be more incremental relative to ongoing ML-for-branching work. Overall, Paper 1 has higher potential breadth and societal relevance.
Paper 2 has higher potential scientific impact because it targets a foundational, timely problem—making responsibility in agentic AI computable via explicit provenance—relevant across many domains (software agents, governance, safety, auditing, law/policy). Its framing (responsibility gaps, causal attribution, responsibility tensor, lifecycle-layer computability, intervention) could set a broad research and standards agenda with significant real-world adoption pressure. Paper 1 is technically novel and useful for RLHF/GRPO efficiency, but its impact is narrower to LLM training and depends on robustness/generalization of probing-based rewards.
Paper 2 has higher likely impact: it targets a broad, urgent, cross-domain problem (responsibility and trust in agentic AI) with a unifying proposal—explicit, computable provenance—supported by formal constructs (causal attribution function, responsibility tensor) and lifecycle framing, plus preliminary experiments and an incident analysis. Its applications span many high-stakes deployments (software engineering and beyond) and align with fast-moving governance needs. Paper 1 is novel and rigorous as a benchmark for scientific task-clarification, but its scope is narrower (computational science domains) and impact is primarily within LLM evaluation rather than system-level accountability.
Paper 2 addresses a fundamental and timely challenge—responsibility and provenance in agentic AI—that has broad implications across AI safety, policy, law, and multiple application domains. It proposes a novel formal framework (causal attribution function, responsibility tensor) for an increasingly urgent problem as agentic AI proliferates. Paper 1, while technically sound with strong empirical results, addresses a narrower problem (LLM evaluation via query clustering) with more limited cross-disciplinary impact. Paper 2's timeliness, breadth of relevance across technical and sociotechnical dimensions, and positioning at the frontier of AI governance give it higher potential impact.
Paper 2 likely has higher scientific impact due to a clearer algorithmic contribution with demonstrated empirical performance across multiple combinatorial optimization benchmarks, open-source code, and a broadly reusable methodological template (latent-space, gradient-based search for program synthesis/heuristic design) that can transfer to other domains. It is timely in automated algorithm design and LLM-based optimization, with direct real-world applications in logistics and operations research. Paper 1 addresses an important governance need, but its impact may hinge on standardization/adoption and appears more conceptual with preliminary experiments.
Paper 1 is likely to have higher scientific impact: it targets a timely, cross-cutting problem (responsibility and trust in agentic AI) with broad relevance across AI safety, governance, HCI, software engineering, and causal attribution. Its focus on explicit, computable provenance and formal constructs (causal attribution function, responsibility tensor) could shape standards and tooling with real-world policy and engineering implications. Paper 2 presents an incremental model improvement in a mature traffic-forecasting GNN literature, with narrower domain impact and likely limited novelty beyond architectural variations.
Paper 1 addresses a fundamental, broadly applicable challenge—responsibility and provenance in agentic AI—that spans multiple domains and has significant policy, legal, and technical implications. Its formalization of responsibility through causal attribution and a responsibility tensor offers novel conceptual infrastructure for an increasingly critical problem as agentic AI proliferates. Paper 2, while technically solid and practically useful for cybersecurity knowledge graph construction, addresses a narrower domain with incremental methodological contributions (task-bank rewards, ontology-guided extraction). Paper 1's breadth of impact, timeliness, and cross-disciplinary relevance give it higher potential scientific impact.
Paper 2 presents a highly novel, counter-intuitive empirical finding—the 'capability paradox'—backed by rigorous, large-scale experiments (42,000 trials). Its identification of semantic hijacking and the proposed heterogeneous ensemble verification offer immediate, actionable solutions for multi-agent system security. While Paper 1 addresses an important conceptual gap in AI responsibility, Paper 2's methodological rigor, quantifiable impact (reducing Attack Success Rate from 52.8% to 2.0%), and concrete real-world applicability give it a significantly higher potential for immediate scientific and practical impact.
Paper 1 has higher potential scientific impact due to greater novelty and breadth: it argues for explicit, computable provenance as a structural requirement for responsible agentic AI, proposes formal constructs (causal attribution function, responsibility tensor), and targets a cross-cutting governance gap relevant across domains using agents (software, healthcare, finance, etc.). Its focus is timely given rapid agent deployment and regulatory pressure, and it could reshape evaluation and accountability practices broadly. Paper 2 is methodologically solid and practically impactful for enterprise document processing, but is more application-specific and closer to systems engineering than a field-shifting scientific agenda.
Paper 1 addresses a critical, timely bottleneck in the widespread deployment of agentic AI: accountability and safety. By formalizing explicit provenance and responsibility, it offers foundational contributions that span machine learning, systems engineering, and AI policy. While Paper 2 presents strong empirical advances in embodied AI, Paper 1 has a broader potential impact across multiple disciplines and the entire AI ecosystem.
Paper 1 demonstrates significantly higher scientific impact due to its rigorous, large-scale real-world deployment and empirical validation. While Paper 2 offers a valuable theoretical framework for AI provenance, Paper 1 presents a production-proven system deployed across thousands of enterprise hosts, addressing immediate, critical security vulnerabilities in Agentic AI. Furthermore, Paper 1 introduces a scalable architecture and a new benchmark (ADR-Bench) that directly enables future empirical research. Its combination of methodological rigor, demonstrated utility at enterprise scale, and tangible open-source contributions gives it a clear edge in actionable, near-term impact.
Paper 2 addresses a fundamental structural problem in agentic AI—the lack of explicit provenance for responsibility attribution—proposing a formal framework (causal attribution function, responsibility tensor) with broad applicability across all agentic AI systems. Its impact extends beyond benchmarking into governance, policy, and system design, touching multiple stakeholder communities. Paper 1, while methodologically rigorous and timely, is primarily an empirical benchmark evaluation of specific current models that will quickly become dated. Paper 2's conceptual contributions have longer-lasting and broader cross-disciplinary impact spanning AI safety, law, and sociotechnical systems.