When Does Routing Become Interpretable? Causal Probes on Block Attention Residuals
Aydin Javadov
Abstract
Block Attention Residuals (Block AttnRes) by replace fixed additive residuals with a learned softmax over earlier depth-source representations, surfacing cross-layer routing as an inspectable tensor in the forward pass. This is a tempting interpretability target: information flow normally inferred indirectly is now directly observable. We ask whether such exposure suffices for mechanistic interpretation. We probe two same-scale (B) Block AttnRes checkpoints under identical routing-ablation interventions: a vanilla Qwen3 inference-wrapped through a deterministic recency-bias schedule that the codebase admits as a routing-equivalent loading path, and a Block AttnRes Qwen3 trained from scratch with routing as part of optimisation. The wrapped baseline's routing weights are content-independent and reproduce the schedule's analytic prediction. The trained AttnRes checkpoint instead exhibits three localised routing motifs: an embedding-source pathway through early-layer MLP, a current-state pathway through early-layer attention and MLP, and an older-history pathway through late-layer attention. Beyond this stratification, we find a sharp dissociation between average routing mass and causal importance: in both sublayers, the largest mass slice is not the largest causal contribution, and one source family carries appreciable mass with no detectable causal role under intervention. Architectural exposure of routing is therefore necessary but not sufficient for mechanistic interpretation: structured depth routing emerges only when routing has been part of training, and even then, descriptive routing summaries should be treated as candidate hypotheses to be tested by causal interventions, not as evidence of mechanism in their own right.
AI Impact Assessments
(1 models)Scientific Impact Assessment
Core Contribution
This paper asks a pointed question about the interpretability of Block Attention Residuals (Block AttnRes), an architecture that replaces fixed additive residual connections with learned softmax routing over earlier depth-source representations. The central thesis is that architectural exposure of routing weights is necessary but not sufficient for mechanistic interpretation. The authors demonstrate this through a controlled comparison of two 0.6B-parameter checkpoints: a vanilla Qwen3 wrapped post-hoc through a deterministic recency-bias schedule (producing content-independent routing weights), and a Block AttnRes Qwen3 trained from scratch with routing as part of optimization. The key finding is a three-way decomposition: (1) only training produces structured routing, (2) trained routing exhibits localized causal motifs, and (3) average routing mass systematically dissociates from causal importance.
Methodological Rigor
The experimental design is clean and well-controlled. The two-checkpoint comparison — same model class, scale, tokenizer, probe, and dataset — isolates the effect of training routing parameters versus merely exposing them architecturally. The mask-and-renormalize ablation framework is a reasonable causal intervention methodology that preserves total mass while testing reliance on specific pathways.
However, there are notable limitations to rigor:
The analytical prediction for the recency-bias schedule matching observed routing mass to three decimal places (0.840) is a satisfying validation of the baseline condition.
Potential Impact
The paper's central message — that descriptive routing summaries should be treated as hypotheses requiring causal validation, not as evidence of mechanism — is a sound methodological principle for the interpretability community. This is particularly relevant as routing-based architectures (Mixture of Experts, Block AttnRes, etc.) proliferate and their routing weights become tempting interpretability shortcuts.
The practical impact is modest for several reasons:
Timeliness & Relevance
The paper is timely in engaging with Block AttnRes shortly after its introduction, and the broader question of when architectural transparency translates to interpretability is increasingly relevant as the field explores alternatives to post-hoc interpretation. The work connects to growing interest in "interpretability by design" — architectures that make internal computations more legible — and provides a cautionary note that legibility ≠ interpretability.
Strengths
1. Clean experimental control: The same-probe, same-model-class comparison is well-designed and the baseline condition provides a useful null model.
2. Important dissociation finding: The mass-vs-causality dissociation (e.g., embedding in MLP carrying less than half of current's mass but five times its causal effect; prev_completed carrying appreciable mass with no detectable causal role) is a concrete, well-demonstrated finding.
3. Clear presentation: The paper is well-written with effective visualizations. The heatmap in Figure 3 and the bar plots make the key findings immediately accessible.
4. Reproducibility: Code is provided, the experimental setup is fully described, and the synthetic dataset is trivially reproducible.
Limitations & Weaknesses
1. Limited generalizability: Single scale, single task, single checkpoint per condition, no naturalistic evaluation. The "three localized motifs" could be specific to this particular training run on this particular data.
2. Modest novelty of the central claim: The principle that descriptive statistics require causal validation is not new to interpretability research. The specific application to Block AttnRes routing is novel but narrow.
3. Weak baseline competence: Both models performing at ~54% on a simple retrieval task suggests these are undertrained or underpowered models, making it unclear whether the routing structure observed is representative of what would emerge in a well-trained, capable system.
4. No mechanistic explanation: The paper identifies *that* certain motifs exist but offers no explanation for *why* they emerge or what computational role they serve beyond loose analogies ("injection point," "read-out point").
5. The paper studies a very specific implementation detail (block-level routing with specific block sizes) rather than establishing principles that would transfer to other routing architectures.
Overall Assessment
This is a competent, clearly presented empirical study that makes a valid methodological point about the gap between architectural transparency and mechanistic interpretability. The experimental design is its strongest feature. However, the scale of investigation is small, the task is simple, the central insight is partially anticipated by existing interpretability methodology, and the specific findings about routing motifs lack theoretical grounding. It represents a useful data point for the community working on routing-based architectures but is unlikely to have broad influence beyond that niche.
Generated Jun 12, 2026
Comparison History (13)
Paper 2 addresses a more broadly impactful topic—interpretability and mechanistic understanding of neural network architectures—which is a central concern across the entire deep learning community. Its finding that architectural exposure of routing is necessary but not sufficient for mechanistic interpretation provides a cautionary, generalizable insight for interpretability research. Paper 1, while technically solid, represents an incremental improvement to a niche method (physics-informed neural particle flow) with a narrower audience primarily in Bayesian filtering/tracking. Paper 2's conclusions about the gap between descriptive summaries and causal mechanisms have wider methodological implications.
Paper 1 addresses a critical bottleneck in AI-driven molecular generation (quality control and uncertainty), offering direct, high-impact applications in drug discovery and materials science. While Paper 2 provides valuable insights into LLM interpretability, its focus on a specific architectural variant (Block AttnRes) makes its immediate scientific impact more niche compared to the broad, interdisciplinary utility of reliable molecular diffusion models.
Paper 2 has higher likely scientific impact due to clear real-world applicability (scalable passive acoustic monitoring for ecology), strong timeliness (biodiversity assessment), and broader downstream utility (embeddings + visualization tool) for ecologists and conservation practitioners. Methodologically, it combines semi-/self-supervised learning, distillation, and active learning with substantial empirical gains over a strong baseline, suggesting robustness and transfer potential. Paper 1 is novel and valuable for mechanistic interpretability research, but its impact is narrower (mainly ML interpretability/architecture analysis) and the contributions are more diagnostic than enabling immediate external applications.
Paper 2 has higher impact potential due to strong real-world applicability (deployed safety classifier monitoring), timely relevance to LLM safety, and higher methodological rigor (pre-registered, large factorial evaluation with CIs and variance decomposition). It offers actionable findings about when conformal adaptation fails (importance-weight collapse) and practical mitigations (dimensionality reduction), with implications for online ML monitoring and reliability engineering. Paper 1 is novel for interpretability of routing and provides useful causal analysis, but its contributions are narrower and more architecture-specific, likely limiting breadth and immediate deployment impact.
Paper 1 addresses a critical reproducibility and comparability crisis in wearable human activity recognition through a massive, standardized benchmark (30 datasets, 17 architectures, 4760 runs) with an open-source framework. Its breadth of impact across applied ML, mobile computing, and health monitoring communities, combined with practical deployment efficiency analysis, gives it wide real-world utility. Paper 2 offers valuable but narrower insights into interpretability of a specific architecture (Block AttnRes), contributing incremental understanding to mechanistic interpretability. Paper 1's benchmark infrastructure and actionable findings for practitioners likely drive broader and more lasting impact.
Paper 1 (Diff-prior) addresses a fundamental limitation in neural relational inference by introducing diffusion-based learnable graph priors, offering broad applicability across NRI architectures and real-world dynamical systems. It provides a novel, principled framework combining diffusion models with variational inference for structure discovery—a widely relevant problem. Paper 2 provides valuable interpretability insights about Block Attention Residuals but is narrower in scope, focused on a specific architecture variant at a single scale, and its conclusions (routing exposure is necessary but insufficient) are somewhat expected. Paper 1's methodological contribution and broader applicability suggest higher impact.
Paper 2 likely has higher scientific impact due to broader applicability and stronger real-world relevance: it introduces a PEFT-like method that works with frozen, precompiled MLLMs and high-throughput inference engines (e.g., vLLM), addressing a practical deployment bottleneck. Optimizing raw visual inputs as a universal adaptation channel is a novel, timely idea with potential cross-domain uses (efficient customization, secure/controlled adaptation, multimodal prompting). Paper 1 is rigorous and valuable for interpretability science but is narrower in application and impact beyond mechanistic interpretability research.
Paper 1 addresses a broadly impactful problem—discovering governing equations from minimal data—relevant across science and engineering. Its active learning strategy for SINDy in ultra-low data regimes has clear practical applications in domains where data acquisition is expensive. The methodology is rigorous, tested on multiple ODE/PDE systems with varying complexity and noise. Paper 2 provides useful interpretability insights for a specific architecture (Block AttnRes) but has narrower scope, addressing a niche architectural question with findings (routing mass ≠ causal importance) that, while valuable, impact a smaller research community.
Paper 1 introduces a novel theoretical framework providing computable, certified predictability horizons for equivariant world models, connecting symmetry structure to Lyapunov spectra with both upper and lower bounds. It demonstrates practical applicability across multiple domains (Lorenz-96, TD-MPC2, V-JEPA) and offers training-free auditing of pretrained models. The theoretical contributions (orbit-constant error characterizing equivariance, budget-aware certificates) are fundamental and broadly applicable to AI safety and deployment. Paper 2 provides useful but more incremental insights about routing interpretability in a specific architecture, with narrower scope and applicability.
Paper 1 likely has higher impact: it introduces a broadly applicable fine-tuning framework for any-length discrete diffusion with a principled measure-theoretic foundation (Radon–Nikodym path-derivative) and provable convergence to reward-tilted distributions without target samples, plus a concrete optimality-guaranteed loss (AJD) and empirical gains. This is timely for controllable generation and can generalize across sequence domains. Paper 2 offers valuable interpretability insights and causal methodology, but its novelty and application scope are narrower and more diagnostic than enabling.