Responsible Agentic AI Requires Explicit Provenance
Jinwei Hu, Xinmiao Huang, Qisong He, Youcheng Sun, Yi Dong, Xiaowei Huang
Abstract
Agentic AI is rapidly proliferating across diverse real-world domains such as software engineering, yet public trust has not kept pace. The central reason is that responsibility, despite being widely discussed, remains a subjective and unenforced concept, as no current agentic framework produces the quantifiable, traceable, and interventionable provenance needed to assign it when harm emerges from compositions no single party designed. We position that what is missing is not better benchmark-level evaluation but across the full agentic lifecycle, which is the only viable basis for making responsibility computable and actionable. We advance this agenda along four axes: establishing such provenance is a structural necessity by identifying responsibility gaps across sociotechnical dimensions, formalizing it must encode through a causal attribution function and responsibility tensor, discussing it can be made computable across four lifecycle layers with preliminary experiments showing that provenance is estimable and interveneable online before irreversible harm accumulates, and examining bears responsibility through a concrete agentic incident. Explicit provenance is not a discretionary refinement but the necessary condition for responsible agentic AI, and no stakeholder across its ecosystem can afford to treat it as optional.
AI Impact Assessments
(1 model)Scientific Impact Assessment
Core Contribution
This paper positions "explicit provenance" as the necessary infrastructure for making responsibility computable and actionable in agentic AI systems. The authors argue that current trustworthy AI approaches, focused on per-component benchmarking and auditing, are structurally insufficient for agentic systems where harm emerges from compositional, multi-step trajectories involving multiple stakeholders. The contribution is organized along four axes: why provenance is necessary (sociotechnical responsibility gaps), what it must encode (a causal attribution function and responsibility tensor), how it can be made computable (a four-layer lifecycle framework with preliminary neuro-symbolic experiments), and who bears responsibility (illustrated through a concrete incident).
The main novelty lies in the formalization attempt—particularly the responsibility tensor R ∈ [0,1]^{|P|×|Ω|×|D|} and the causal contribution function κ(p, ω, τ)—which tries to bridge the gap between philosophical responsibility discourse and computational operationalization. The four-layer lifecycle framework (Design, Engineering, Deployment, Experience) provides a structured research agenda.
Methodological Rigor
The paper operates primarily as a position paper with preliminary empirical support. The formalizations, while conceptually motivated, raise significant concerns:
The causal contribution function κ(p, ω, τ) = Pr[ω|τ] − Pr[ω|τ_{−p}] relies on counterfactual trajectories that are acknowledged to be theoretical anchors rather than computable quantities. The paper does not adequately address the fundamental problem that counterfactual reasoning in complex multi-agent systems is computationally intractable in general. The gap between the formal definition and what the experiments actually measure (AUPRC of failure prediction from execution prefixes) is substantial—predicting failure from prefixes is not the same as computing counterfactual causal contributions.
The responsibility tensor is formally defined but its practical instantiation remains largely illustrative. The dimension weights w_k are acknowledged as requiring human judgment, which somewhat undermines the "computable" framing. The completeness condition (responsibilities sum to 1) is a normative choice presented as a formal requirement without sufficient justification for why responsibility should be zero-sum.
The preliminary experiments (Section 5.5) demonstrate that neuro-symbolic monitors can predict trajectory failure from execution prefixes across four benchmarks, with AUPRC substantially above random baselines. This is a meaningful empirical contribution, but the connection to the broader provenance framework is indirect. Detecting that something is going wrong is not equivalent to attributing causal responsibility to specific parties—a significant conceptual leap that the paper acknowledges but does not bridge.
The Example 1 (WebArena responsibility assignment) is illustrative rather than validated. The mapping from DFA states to responsibility assignments involves substantial interpretive judgment that the paper presents as more mechanical than it actually is.
Potential Impact
The paper addresses a genuinely important problem at the intersection of AI safety, governance, and deployment. Several aspects could influence the field:
1. Framing contribution: The articulation that responsibility requires three simultaneous properties (quantifiability, traceability, interventionability) provides a useful analytical framework for the responsible AI community.
2. Research agenda: The four-layer lifecycle framework (L1-L4) identifies concrete research directions that could organize future work, particularly around compositional verification and population-scale monitoring.
3. Bridge-building: The paper attempts to connect technical AI safety with legal, ethical, and regulatory frameworks, which is valuable given the current policy moment around AI governance (EU AI Act, AI Liability Directive).
4. Practical monitoring: The neuro-symbolic monitoring approach, while preliminary, suggests a viable path toward runtime provenance that could be adopted in production agentic systems.
However, the impact is tempered by the significant gap between the formal framework and its practical realizability. The paper risks setting up an impossibly high standard (full counterfactual causal attribution) that may discourage rather than enable practical progress.
Timeliness & Relevance
The paper is highly timely. Agentic AI deployment is accelerating rapidly (as evidenced by the industry surveys cited), and the governance gap is real. The EU AI Liability Directive's application to agentic systems is an active policy question, and frameworks for attributing responsibility in compositional AI systems are urgently needed. The paper correctly identifies that this is a structural problem rather than a matter of incremental improvement to existing evaluation paradigms.
Strengths
Limitations
Overall Assessment
This is a well-motivated position paper that identifies a genuine and important problem in agentic AI governance. Its primary contribution is conceptual framing rather than technical innovation. The formal framework is ambitious but the gap between formalism and empirical validation is significant. The paper would benefit from a more honest assessment of what is currently achievable versus aspirational, and from deeper engagement with the computational complexity of counterfactual reasoning in multi-agent systems.
Generated May 19, 2026
Comparison History (23)
Paper 1 addresses a foundational bottleneck in AI deployment—accountability and trust—by proposing a formal framework for explicit provenance. Its focus on sociotechnical safety, causal attribution, and regulatory alignment gives it broader cross-disciplinary and societal impact compared to Paper 2, which offers a narrower, albeit valuable, algorithmic improvement for multi-agent reasoning capabilities.
GeoX presents a novel, concrete technical framework combining self-play reinforcement learning with executable program verification for geospatial reasoning—a growing field with significant real-world applications (remote sensing, urban planning, disaster response). It demonstrates measurable improvements (5.5 points) over strong baselines without requiring large-scale human annotation, and releases a benchmark. Paper 2, while addressing an important topic (responsible AI provenance), is primarily a position/framework paper with only preliminary experiments. Its impact depends on adoption of proposed frameworks, which historically faces challenges. GeoX's methodological contribution and empirical results offer more immediate and citable scientific impact.
Paper 1 introduces a concrete algorithmic improvement (DMPO) to solve a critical bottleneck (mode collapse) in highly relevant RL methods like GRPO. Its strong empirical validation across diverse reasoning tasks suggests immediate and widespread utility in the rapidly advancing field of LLM reasoning. While Paper 2 addresses vital AI safety concerns, its position-paper nature and conceptual focus will likely yield less direct, measurable scientific impact and fewer follow-up algorithmic innovations compared to Paper 1.
Paper 2 offers higher immediate scientific impact due to its strong methodological rigor and concrete solutions to a critical bottleneck in modern AI: LLM training instability and compute waste. While Paper 1 addresses an important conceptual gap in AI governance, it is primarily a position paper with preliminary experiments. In contrast, Paper 2 provides a novel algorithmic intervention (LBW-Guard) supported by extensive empirical validation across multiple LLM scales (up to 14B). Its ability to maintain training stability under extreme stress has massive, immediate real-world applications for reducing the exorbitant costs of foundation model training.
Paper 1 addresses a universal and critical bottleneck in the deployment of agentic AI systems across all domains. By introducing a formal framework for explicit provenance and a computable responsibility tensor, it offers a foundational, cross-disciplinary solution that spans software engineering, AI ethics, and law. While Paper 2 provides highly valuable empirical safety data for autonomous driving, Paper 1's conceptual framework has broader potential to shape the fundamental architecture and regulation of future multi-agent AI ecosystems.
Paper 1 presents a practical, validated methodology (Noise2Noise denoising for Raman spectroscopy) with concrete experimental results demonstrating significant speedup (5ms acquisitions) while maintaining spectral fidelity. It addresses a real bottleneck in high-throughput spectroscopy with a transferable framework. Paper 2 is a position/framework paper on AI provenance that, while timely, lacks substantial empirical validation and operates in a crowded responsible-AI discourse space. Paper 1's concrete methodological contribution and reproducible pipeline are more likely to generate direct citations and adoption across spectroscopic communities.
Paper 1 addresses a fundamental and increasingly critical problem—responsibility and provenance in agentic AI—with novel formal contributions (causal attribution function, responsibility tensor) that have broad implications across AI safety, governance, policy, and multiple application domains. Its framework-agnostic, structural approach to a widely recognized trust gap gives it high potential for cross-disciplinary impact and policy relevance. Paper 2, while practically useful, presents an incremental engineering contribution analyzing specific interaction paradigms within a single framework (buddyMe), with narrower scope and limited generalizability beyond its specific system context.
Paper 2 tackles a critical, overarching challenge in AI—accountability and safety in agentic systems—by proposing a formal framework for explicit provenance. This addresses urgent sociotechnical and ethical needs, offering broad, cross-disciplinary impact spanning AI, policy, and engineering. While Paper 1 offers a valuable benchmark for evaluating specific spatial and temporal reasoning capabilities in LLMs, its scope and applications are significantly narrower than the foundational safety and responsibility framework proposed in Paper 2.
Paper 1 demonstrates higher scientific impact through its concrete, large-scale empirical validation (3,000 trials, 119 knowledge infrastructures across 14 Earth-science domains) and addresses a critical practical barrier—democratizing access to process-based simulation models. It delivers a generalizable toolkit (KDT) with measurable performance gains (84% vs 40% success). Paper 2, while addressing an important problem (provenance for responsible AI), is primarily a position/framework paper with only preliminary experiments. Paper 1's combination of methodological rigor, immediate practical utility, and breadth across Earth sciences gives it stronger near-term and long-term impact potential.
Paper 1 targets a cross-cutting, timely bottleneck for deploying agentic AI safely: making responsibility computable via explicit, lifecycle-wide provenance. If adopted, its concepts (causal attribution, responsibility tensor, online interveneable provenance) could influence standards, auditing, governance, and system design across many application domains, yielding broad, long-lived impact. Paper 2 is technically strong and practically valuable for MILP, but its impact is narrower to optimization/solver communities and may be more incremental relative to ongoing ML-for-branching work. Overall, Paper 1 has higher potential breadth and societal relevance.