Beyond Post-hoc Explanation: Toward Glassbox AI via Probabilistic Mediation
Manuele Leonelli
Abstract
Large language models are rapidly becoming infrastructural components in high-stakes institutional settings, including public administration, legal reasoning, and healthcare, where opacity is not merely inconvenient but institutionally and legally untenable. Existing approaches to explainability are predominantly post-hoc, offering unstable, non-contestable accounts that have no formal relationship to the reasoning process that produced the output. We argue that the problem is not the absence of explanation but the absence of structured reasoning in the first place. This paper makes the case for a fundamentally different architecture, which we call the Glassbox Framework, in which Bayesian networks serve as transparent, ante-hoc mediation layers for generative models. Bayesian networks encode domain knowledge, causal assumptions, and probabilistic dependencies before inference occurs, enabling auditable reasoning traces, uncertainty quantification, and contestable outputs. We characterise the architecture of this framework and ground it in a benefit eligibility scenario, identifying the foundational challenges spanning semantic alignment, dynamic model construction, probabilistic grounding, and human governance that must be solved to realise it at scale. By shifting from post-hoc explanation to ante-hoc probabilistic mediation, this work outlines a principled path toward AI systems that are not only powerful but fundamentally accountable.
AI Impact Assessments
(1 models)Scientific Impact Assessment
Core Contribution
The paper proposes the "Glassbox Framework," a conceptual architecture in which Bayesian networks (BNs) serve as ante-hoc probabilistic mediation layers between large language models (LLMs) and consequential institutional outputs. The core argument is that post-hoc explainability (LIME, SHAP, etc.) is structurally inadequate for high-stakes settings because it has no formal relationship to the model's actual reasoning process. Instead, the authors propose inserting a transparent, formally specified BN between the LLM's language processing capabilities and the final decision output, so that reasoning is structured and inspectable *before* inference occurs rather than approximated afterward.
The framework is organized into three layers: a governance layer (institutional specification and maintenance of the BN), an inference layer (iterative LLM-BN interaction mediated by a semantic translation interface), and an accountability layer (audit trails and contestation mechanisms). The paper identifies five foundational research challenges: semantic grounding, structure learning in normative domains, the probabilistic interface between LLMs and BNs, human-in-the-loop governance, and scalability across domains.
Methodological Rigor
This is a position/vision paper, not an empirical contribution, and must be evaluated accordingly. As such, it presents no experiments, no implemented system, no quantitative evaluations, and no formal proofs. The benefit eligibility scenario (Table 2, Figure 3) is illustrative rather than empirical — the posterior probabilities shown are hypothetical, and no actual LLM-BN pipeline was constructed or tested.
The paper is honest about this limitation, explicitly stating "this paper does not claim to have built this system." However, this honesty does not compensate for the absence of even a minimal proof-of-concept. The proposed architecture involves several interacting components (semantic translation, virtual evidence injection, iterative re-querying, subgraph selection), and without any implementation, it is impossible to evaluate whether the framework is technically feasible, whether the components compose coherently, or whether the claimed properties (stability, contestability, native uncertainty quantification) actually materialize in practice.
The critique of post-hoc explainability is well-articulated and draws on established literature (Rudin 2019, Alvarez Melis and Jaakkola 2018, Miller 2019), though it largely rehearses known arguments rather than advancing new analytical results. The three failure modes (instability, non-contestability, accountability gap) are appropriately identified but not novel observations.
Potential Impact
The paper's framing addresses a genuinely important problem: the deployment of LLMs in high-stakes institutional settings where legal and ethical accountability is required. The idea of using BNs as structured mediation layers is conceptually appealing and aligns with real regulatory demands (EU AI Act). If realized, such an architecture could meaningfully advance accountability in automated public administration, legal reasoning, and healthcare.
However, the gap between the vision and realizability is substantial. The paper identifies five open research challenges, and the authors themselves acknowledge that several of these — particularly semantic grounding and the probabilistic interface — lack adequate existing solutions. The semantic alignment problem illustrated in Figure 2 is particularly well-characterized, showing how natural language representations resist clean mapping to discrete BN variables. But characterizing a problem is not the same as solving it, and the paper offers no concrete technical paths toward solutions.
The framework also faces a fundamental scalability concern that the paper acknowledges but does not resolve: real-world high-stakes domains (criminal law, complex medical reasoning, multi-jurisdictional administrative decisions) involve hundreds or thousands of relevant variables with intricate dependencies. Whether expert-elicited BNs can capture this complexity while remaining genuinely inspectable and contestable is far from clear.
Timeliness & Relevance
The paper is timely. The rapid deployment of LLMs in institutional settings, combined with the EU AI Act's requirements for transparency and auditability in high-risk applications, creates genuine demand for architectures that go beyond post-hoc explanation. The framing of "ante-hoc accountability" addresses a real conceptual gap in current discourse. The paper also draws attention to the social dimension of the problem — that populations most exposed to AI-driven decisions are least equipped to contest them — which grounds the technical argument in important equity considerations.
Strengths
1. Clear problem framing: The distinction between post-hoc explanation and ante-hoc structured reasoning is clearly drawn and conceptually valuable.
2. Honest identification of open problems: The paper does not oversell. The five foundational challenges are specific, well-characterized, and constitute a useful research agenda.
3. Virtual evidence specification: The argument that LLM outputs should enter the BN as virtual evidence (likelihood ratios) rather than hard evidence is a technically precise design choice that demonstrates thoughtful engagement with the probabilistic machinery.
4. Governance as first-class concern: Treating the governance layer as an institutional component (not merely a technical one) and recognizing that governance failure propagates silently through the system is a mature design insight.
5. Bootstrap circularity identified: The recognition that subgraph selection cannot depend on LLM parsing without circularity, and the proposed conservative initialization strategy, shows careful architectural thinking.
Limitations
1. No implementation or empirical validation: The framework is entirely conceptual. The illustrative scenario uses hypothetical numbers and no actual LLM-BN interaction was tested.
2. Limited novelty in the component ideas: Using BNs for structured reasoning and combining them with neural models has a long history in neuro-symbolic AI. The paper distinguishes itself from neuro-symbolic integration but the distinction is more rhetorical than technical — the actual proposed architecture (LLM extracts evidence, feeds it to a graphical model, iterates) shares substantial overlap with existing hybrid systems.
3. Underengagement with related work: The paper does not engage with the substantial literature on neuro-symbolic AI, probabilistic programming languages, or structured prediction with graphical models that addresses overlapping concerns. Frameworks like DeepProbLog, Pyro, or various LLM-as-reasoner architectures are not discussed.
4. Self-citation density: Several citations are to the authors' own work, including survey-type results that are tangential to the core technical contribution.
5. Scalability concerns unresolved: The acknowledgment that each domain requires a custom BN, with no demonstrated transfer mechanism, severely limits practical impact.
6. The "abstraction trap" is acknowledged but not addressed: The paper notes that encoding normative reasoning into BNs risks losing social context (Selbst et al. 2019) but offers no strategy to mitigate this.
Overall Assessment
This is a well-written position paper that articulates a compelling vision for accountable generative AI through probabilistic mediation. Its strength lies in problem framing and research agenda definition rather than in technical contribution. The identified challenges are genuine and specific, which is valuable for directing future research. However, the absence of any implementation, formal analysis, or empirical validation limits the paper's immediate scientific impact. The contribution is primarily discursive: it reframes the explainability debate in useful ways but does not advance the technical state of the art.
Generated Jun 8, 2026
Comparison History (17)
Paper 2 presents a concrete, empirically validated method (PCI) that achieves state-of-the-art results on TSP benchmarks with measurable improvements in optimality gap, inference time, and memory usage. It offers a practical, retraining-free contribution to neural combinatorial optimization. Paper 1, while addressing an important problem (AI transparency), is primarily a conceptual/position paper proposing a framework without empirical validation. Its Glassbox Framework identifies challenges but doesn't solve them, limiting near-term scientific impact. Paper 2's concrete algorithmic contribution with reproducible results is more likely to drive immediate follow-up research and adoption.
Paper 1 presents a concrete, novel training method (uncertainty-aligned RL reward shaping plus lightweight annotations) with benchmarked improvements in tool-use decisions and calibrated uncertainty—likely to be adopted and extended in current agentic LLM pipelines. Its methodological rigor and near-term applicability to widely deployed tool-calling agents support strong impact. Paper 2 is timely and potentially broad (governance/accountability) but is primarily conceptual/architectural with limited demonstrated implementation, making near-term scientific uptake and measurable influence less certain despite high-level importance.
Paper 2 addresses the critical, widespread problem of LLM opacity in high-stakes domains (healthcare, law) by proposing a novel ante-hoc explainability framework. Its integration of Bayesian networks for structured, auditable reasoning offers a more principled path to accountability than current post-hoc methods. While Paper 1 is highly innovative in agent memory, Paper 2's potential to enable the safe deployment of LLMs in regulated, real-world settings gives it a broader and more urgent scientific and societal impact.
Paper 1 presents a concrete, operational methodology (RWSA + W2S) for automatically constructing executable Skills from real interaction traces, with empirical evaluation on 70 Skills and measurable gains over baselines—supporting methodological rigor, near-term applicability, and timeliness for LLM agent engineering. Paper 2 is a compelling position/architecture proposal for glassbox AI via Bayesian mediation with broad societal relevance, but it is largely conceptual with limited demonstrated implementation or evaluation, making its scientific impact more uncertain despite potentially high long-run influence.
Paper 2 presents a concrete, implemented multi-agent system with empirical state-of-the-art results on established benchmarks, demonstrating immediate practical impact in the rapidly growing Deep Research paradigm. Paper 1, while intellectually compelling in arguing for ante-hoc probabilistic mediation via Bayesian networks, remains a conceptual/position paper outlining a framework without implementation or empirical validation. Paper 2's specific architectural innovations (recursive search agents, rubric-grounded reasoning, graph-based planning) are immediately actionable and reproducible, giving it broader near-term scientific influence despite Paper 1's important theoretical contribution to AI accountability.
Paper 2 presents a concrete, executable benchmark with empirical results evaluating frontier LLMs on interactive reasoning, metacognition, and robustness—filling a clear gap in LLM evaluation methodology. It provides reproducible, quantitative findings across 474 games and multiple models. Paper 1, while addressing an important problem (AI transparency), is primarily a conceptual/architectural proposal without implementation or empirical validation, reducing its immediate scientific impact. Paper 2's benchmark can be widely adopted by the community, driving measurable progress, whereas Paper 1's Glassbox Framework remains speculative.
CogManip provides a concrete, empirical benchmark with validated datasets (1,000 scenarios), systematic evaluation of 13 models including frontier systems, and quantitative findings about manipulation risks. It addresses a timely AI safety concern with a reusable tool the community can build upon. Paper 1, while intellectually interesting, is primarily a conceptual/position paper proposing a framework (Glassbox) without implementation or empirical validation. Its impact depends on future work to realize the architecture. Paper 2's immediate practical utility, empirical rigor, and alignment with urgent AI safety priorities give it higher near-term scientific impact.
Paper 2 proposes a fundamental architectural shift (Glassbox Framework) for AI transparency using Bayesian networks as ante-hoc mediation layers, addressing the critical and widely-relevant problem of AI explainability in high-stakes settings. Its breadth of impact spans public administration, legal reasoning, healthcare, and AI governance—touching regulatory and policy domains. While Paper 1 makes a solid contribution with the SoCRATES benchmark for LLM mediation evaluation, it addresses a narrower problem. Paper 2's conceptual framework has greater potential to influence multiple fields and reshape how AI accountability is architected.
Paper 1 proposes a broadly applicable, timely architectural shift from post-hoc explainability to ante-hoc, auditable reasoning via Bayesian-network mediation for LLMs in high-stakes domains. If realized, it could impact AI governance, legal/health/public-sector deployment, causal/probabilistic modeling, and interpretability, with significant real-world implications. Paper 2 is a solid, more narrowly scoped methodological contribution to process rewards for GUI-agent RL with modest reported gains and impact primarily within embodied/GUI RL benchmarks. Overall, Paper 1 has larger cross-field and societal impact potential despite being more conceptual.
Paper 1 presents a concrete, empirically validated architecture that addresses a critical bottleneck in current AI research (long-horizon memory for LLM agents). While Paper 2 offers a strong conceptual paradigm shift for explainability, Paper 1 demonstrates immediate methodological rigor and measurable improvements over existing baselines, likely leading to faster adoption and citation within the rapidly growing field of autonomous agents.