Learning Quantifiable Visual Explanations Without Ground-Truth
Amritpal Singh, Andrey Barsky, Mohamed Ali Souibgui, Ernest Valveny, Dimosthenis Karatzas
Abstract
Explainable AI (XAI) techniques are increasingly important for the validation and responsible use of modern deep learning models, but are difficult to evaluate due to the lack of good ground-truth to compare against. We propose a framework that serves as a quantifiable metric for the quality of XAI methods, based on continuous input perturbation. Our metric formally considers the sufficiency and necessity of the attributed information to the model's decision-making, and we illustrate a range of cases where it aligns better with human intuitions of explanation quality than do existing metrics. To exploit the properties of this metric, we also propose a novel XAI method, considering the case where we fine-tune a model using a differentiable approximation of the metric as a supervision signal. The result is an adapter module that can be trained on top of any black-box model to output causal explanations of the model's decision process, without degrading model performance. We show that the explanations generated by this method outperform those of competing XAI techniques according to a number of quantifiable metrics.
AI Impact Assessments
(1 model)Scientific Impact Assessment
1. Core Contribution
This paper makes two interrelated contributions to explainable AI (XAI) for visual models. First, it proposes Minimality-Sufficiency Integration (MSI), a perturbation-based evaluation metric for saliency maps that jointly accounts for the sufficiency of highlighted regions (do they contain enough information for prediction?) and their minimality (are they compact?). Second, it introduces Learnable Adapter eXplanation (LAX), a lightweight module trained on top of frozen black-box models to produce saliency maps that optimize a differentiable approximation of MSI, using information bottleneck principles.
The problems addressed — how to evaluate XAI methods without ground truth, and how to generate explanations that are both faithful and parsimonious — are genuine and important. The paper identifies two specific failure modes of existing metrics: insensitivity to overly large masks and poor handling of inputs with multiple valid explanatory regions. MSI attempts to address both via its base score (sufficiency/discriminativeness) and mask penalty (minimality) components.
2. Methodological Rigor
The methodology has several strengths but also notable gaps:
Strengths:
Weaknesses:
3. Potential Impact
The paper addresses a real pain point in XAI — the lack of standardized, reliable evaluation metrics. If MSI proves robust across diverse settings, it could serve as a useful complementary metric. The LAX framework's model-agnostic, adapter-based design is appealing for practical deployment, as it avoids retraining the base model.
However, the impact may be limited by several factors:
4. Timeliness & Relevance
The paper addresses a timely topic — XAI evaluation — which is indeed an open problem receiving increasing attention (e.g., IDSDS at NeurIPS 2024, F-fidelity at ICLR 2025). The information bottleneck framing connects to a well-established theoretical tradition. However, the paper's scope feels somewhat behind the frontier: it focuses on ResNet18 and ViT-B/16 on CIFAR-10-level datasets, while the field is moving toward explaining much larger and more complex models. The lack of engagement with recent large-scale XAI benchmarks or foundation model explainability limits its relevance to current practice.
5. Strengths & Limitations
Key Strengths:
Key Limitations:
Additional Observations
Generated May 19, 2026
Comparison History (17)
Paper 1 addresses a critical bottleneck in current LLM deployment (memory and context window limits) by empirically validating weight-based consolidation over context compaction. Its practical approach to continuous personalization, combined with strong statistical results and a valuable methodological finding regarding validation cross-entropy, gives it high potential to directly influence next-generation LLM architectures.
Paper 2 likely has higher impact due to its large-scale, multinational benchmark with expert calibration, directly addressing urgent safety and reliability gaps for LLMs in real clinical settings. It offers an immediately reusable resource for broad evaluation, risk analysis, and regulatory/clinical adoption discussions, with clear real-world implications (patient harm) and strong timeliness given rapid LLM deployment. Paper 1 is innovative in XAI evaluation and training explanations without ground truth, but its impact is more methodological and narrower, and would depend on adoption and validation across diverse tasks.
Paper 1 addresses a timely and high-impact problem in LLM safety alignment, proposing a modular framework for context-dependent safety control. Given the massive deployment of LLMs and the urgent need for flexible safety mechanisms in professional settings, this work has broader immediate applicability and relevance. Its modular composition approach enabling on-demand multi-domain authorization is novel and practical. Paper 2 contributes meaningfully to XAI evaluation metrics but addresses a more established problem space with comparatively narrower impact scope.
PEEK addresses the timely and high-impact problem of improving LLM agents operating over long contexts, which is central to the rapidly growing field of AI agents. It introduces a novel concept (context maps as orientation caches), demonstrates strong empirical gains across multiple benchmarks and architectures including production systems (OpenAI Codex), and offers practical efficiency improvements (lower cost, fewer iterations). Paper 2 contributes a useful XAI metric and method, but operates in a more mature and narrower subfield with less transformative potential. PEEK's broader applicability to the booming LLM agent ecosystem gives it higher impact potential.
Paper 2 likely has higher impact due to timeliness and broad applicability: agentic LLMs are a rapidly expanding area, and a modular framework for executable, reusable skills can transfer across many tasks (web, math, coding) and deployment settings (inference-time guardrails, post-training supervision, self-improvement). The reported large empirical gains on strong baselines suggest practical relevance. Paper 1 addresses an important XAI evaluation gap and proposes a novel metric plus adapter, but XAI impact is often constrained by domain-specific validation needs and slower adoption compared to agent frameworks.
Paper 2 proposes a novel, actionable framework (metric + method) for evaluating and generating XAI explanations without ground truth, addressing a widely recognized gap in explainable AI. It offers concrete technical contributions—a differentiable metric and a trainable adapter module—with broad applicability across deep learning models. Paper 1 makes a valuable conceptual argument about alignment evaluation levels but is more of a position/audit paper with narrower methodological novelty. Paper 2's technical contributions are more likely to be adopted, cited, and built upon across multiple ML application domains.
Paper 1 addresses a fundamental bottleneck in Explainable AI—the lack of ground truth for evaluation—by introducing a novel, differentiable metric and a universally applicable adapter for black-box models. This methodological innovation has broad implications across all deep learning applications requiring trustworthiness, offering high real-world utility. While Paper 2 provides a valuable diagnostic benchmark for LLMs, Paper 1's foundational contribution to generating and quantifying causal explanations presents a more universally transformative approach to responsible AI deployment.
Paper 2 likely has higher impact due to its large, community-reusable benchmark (52k items from 103 textbooks) and the added dependency graph enabling new evaluation axes. Benchmarks often catalyze broad progress across NLP, theorem proving, and AI-for-math, with clear real-world relevance to verified mathematics and formal methods. Its rigor is supported by scale, careful construction, and strong baseline experiments revealing clear gaps (low accuracy, dependency-depth degradation). Paper 1 is valuable for XAI evaluation and training, but its impact may be narrower and more sensitive to assumptions about perturbation-based causal metrics.
Paper 2 has higher likely impact due to its broad, timely synthesis of dynamic, intervention-aware clinical prediction—directly targeting major real-world deployment barriers (treatment confounding feedback, informative/irregular observation, identifiability). As a unifying framework/review bridging forecasting, counterfactual trajectories, and policy evaluation with concrete evaluation/validation guidance, it can influence multiple subfields (clinical ML, causal inference, time-series modeling, health policy) and shape standards for “decision-grade” evidence. Paper 1 is novel and useful for XAI evaluation, but is narrower in application scope and ecosystem-level influence.
Paper 2 addresses a fundamental challenge in Explainable AI (XAI) by providing a quantifiable evaluation metric and generation method without requiring ground truth. Its contributions offer broad applicability across numerous deep learning domains (e.g., healthcare, autonomous driving), leading to extensive scientific impact. In contrast, Paper 1, while highly rigorous and commercially valuable, is relatively constrained to the specific domain of e-commerce web agents.