Bias by Necessity: Impossibility Theorems for Sequential Processing with Convergent AI and Human Validation
Jikun Wu, Dongxin Guo, Siu-Ming Yiu
Abstract
Are certain cognitive biases mathematically inevitable consequences of sequential information processing? We prove that primacy effects, anchoring, and order-dependence are architecturally necessary in autoregressive language models due to causal masking constraints. Our three impossibility theorems establish: (1) primacy bias arises from asymmetric attention accumulation; (2) anchoring emerges from sequential conditioning with provable information bounds; and (3) exact debiasing by permutation marginalization requires factorial-time computation, with Monte Carlo approximation feasible at constant per-tolerance overhead. We validate these bounds across 12 frontier LLMs (; BIC vs. next-best alternative). We then derive quantitative predictions from the framework and test them in two pre-registered human experiments ( analyzed). Study 1 confirms anchor position modulates anchoring magnitude (, BF). Study 2 shows working memory load amplifies primacy bias (, BF), with WM capacity predicting bias reduction (). These convergent findings reframe cognitive biases as resource-rational responses to sequential processing.
AI Impact Assessments
(1 models)Scientific Impact Assessment
Core Contribution
This paper attempts to establish that primacy bias, anchoring, and order-dependence are mathematically inevitable in autoregressive language models, framing them as "impossibility theorems." The paper then draws computational-level parallels to human cognition, testing predictions in two pre-registered behavioral experiments. The ambition is notable: bridging formal properties of transformer architectures with cognitive science through resource-rationality arguments.
Methodological Rigor
Theoretical claims require scrutiny. The three "impossibility theorems" are substantially less powerful than the framing suggests:
Theorem 1 essentially proves that if a model's output depends on both content and position (Assumption A3), then permuting inputs changes outputs. This is nearly tautological — A3 essentially assumes position-dependence, and the theorem concludes position-dependence exists. The theorem establishes that *some* input pair produces different outputs under reversal, but says nothing about the magnitude or practical significance of primacy bias for typical inputs. The gap between "there exist inputs where order matters" and "primacy bias is inevitable in practice" is enormous.
Theorem 2 relies on a "first-order linearization" (Equation 6-7) that decomposes the hidden state into an anchor-free component plus additive attention terms. This decomposition is an approximation, not exact — transformer hidden states involve layer normalization, residual connections, and nonlinear interactions that make this additive decomposition questionable. The claimed lower bound on mutual information is heuristic rather than rigorous. The "constructive bound" of I_min ≥ 3.2 nats using rough parameter estimates is an order-of-magnitude calculation, not a proven bound.
Theorem 3 is the most straightforward: exact permutation marginalization requires n! forward passes. This is correct but essentially trivial — it's a direct consequence of the definition of permutation marginalization. The Monte Carlo approximation result (ε ≤ C/√k) is standard concentration inequality application.
LLM validation reports R² = 0.89 for an exponential decay model across 12 LLMs, which is compelling model comparison evidence (ΔBIC = 16.6). However, critical details are underspecified: how exactly was "anchoring bias" operationalized across different models? How were the 200 trials per model structured? The prospective validation on Mistral-Large (38.4% observed vs. 40.1% predicted) is a nice touch but represents a single data point.
Human experiments are reasonably well-designed with pre-registration, adequate sample sizes, attention checks, and Bayesian analyses. Study 1 (d = 0.52, BF₁₀ = 847) demonstrates anchor position effects, and Study 2 (d = 0.41, BF₁₀ = 156) shows WM load amplification. However, the claim that these validate the *formal framework* is overstated. The prediction that anchors presented first produce stronger effects than anchors presented later is well-established in the anchoring literature (it's essentially the primacy effect). The "prediction" of d ∈ [0.35, 0.55] involves a calibration constant κ fitted from prior anchoring literature and an empirical correction γ ≈ 1.2 — making this more of a post-hoc calibration than a genuine a priori prediction from first principles.
Potential Impact
The paper's strongest contribution is conceptual: explicitly connecting architectural properties of transformers to cognitive bias literature through Marr's levels framework. This framing could influence:
1. LLM evaluation methodology: The argument that position bias is architectural rather than fixable could shift how LLM-as-judge systems are designed (toward permutation averaging rather than debiasing training).
2. Human-AI collaboration: The suggestion to pair systems with complementary architectural biases is actionable.
3. Cognitive science: Resource-rationality arguments for why biases persist gain formal support, though the human-transformer analogy remains at the computational level with acknowledged mechanistic differences.
Timeliness & Relevance
The paper addresses a timely intersection: position bias in LLMs is a recognized practical problem (Liu et al., 2024; Shi et al., 2025), and the cognitive science community is actively interested in LLM-human parallels. The resource-rationality framing connects to an active research program.
Strengths
Limitations
Overall Assessment
This paper presents an ambitious cross-disciplinary framework with genuine conceptual value, but the mathematical claims are significantly overstated relative to their actual content. The "impossibility theorems" label is misleading — these are observations about properties of sequential processors, dressed in theorem-proof format. The empirical work is solid but the human experiments largely confirm well-known phenomena rather than generating surprising predictions. The paper's greatest value lies in its conceptual framing and practical implications for LLM evaluation design, rather than in its formal contributions.
Generated May 12, 2026
Comparison History (20)
Paper 2 likely has higher scientific impact due to a clearer, immediate real-world application (AI-assisted discovery of new formal proofs) and broad cross-field relevance to mathematics, automated reasoning, and AI tooling. Solving open Erdős problems and OEIS conjectures provides concrete, high-salience milestones, and deployment across multiple research areas suggests scalable impact. Paper 1 is novel and methodologically strong, but its impact is more conceptual/theoretical and may diffuse across cognitive science/ML without comparable headline breakthroughs or direct utility.
Paper 1 establishes fundamental impossibility theorems linking cognitive biases to sequential processing constraints, bridging AI and cognitive science with formal proofs validated across 12 LLMs and pre-registered human experiments. Its novelty lies in proving biases are architecturally inevitable rather than mere flaws, with broad implications for AI alignment, cognitive science, and decision-making. Paper 2 is a strong methodological contribution to automated discovery with principled SMC foundations, but is more incremental within the program synthesis/LLM evolution space. Paper 1's cross-disciplinary theoretical insights and convergent human-AI validation give it greater breadth and lasting impact.
Paper 1 establishes fundamental mathematical theorems linking AI architectural constraints to human cognitive biases, bridging computer science, psychology, and cognitive science. Its theoretical depth, impossibility theorems, and cross-disciplinary validation offer broader foundational impact than Paper 2, which presents a valuable but narrower methodological improvement in multi-agent LLM systems.
Paper 2 bridges AI, mathematics, and cognitive psychology with rigorous impossibility theorems proving cognitive biases are inevitable in sequential processing. It combines deep theoretical proofs with extensive empirical validation across frontier LLMs and pre-registered human experiments. While Paper 1 offers a highly practical evaluation framework for LLM agents, Paper 2 provides a fundamental scientific contribution that reshapes our understanding of cognition in both humans and machines, giving it a profound interdisciplinary impact.
Paper 2 has higher potential impact due to its broader interdisciplinary reach, connecting formal impossibility theorems about LLM architectures to fundamental cognitive science through pre-registered human experiments. It reframes cognitive biases as mathematically inevitable consequences of sequential processing—a profound theoretical insight spanning AI, cognitive science, and psychology. The combination of rigorous mathematical proofs, validation across 12 frontier LLMs, and convergent human behavioral evidence creates a unified framework with deep implications for both AI design and understanding human cognition. Paper 1, while valuable, addresses a more specific diagnostic problem in omnimodal LLMs.
Paper 2 offers fundamental impossibility theorems bridging AI and cognitive science, proving that certain biases are inevitable in sequential processing. Its combination of theoretical proofs, LLM evaluation, and human experiments provides a high-impact, cross-disciplinary contribution. In contrast, Paper 1 presents a valuable but more specialized algorithmic improvement for tool-use in language models.
Paper 2 has higher potential impact due to stronger novelty (impossibility theorems linking architectural constraints to bias), broader cross-field relevance (ML theory, AI safety/interpretability, cognitive science, HCI), and clearer generalizable implications for autoregressive systems. It combines formal results with large-scale validation across many frontier LLMs and preregistered human experiments, indicating methodological rigor and timely relevance. Paper 1 is useful and actionable for single-cell ML practice, but its contributions are more domain-specific and incremental (layer selection guidance) with narrower breadth beyond computational biology.
Paper 1 likely has higher impact because it reveals a simple, high-leverage jailbreak-like mechanism (“stay consistent with prior history”) that can flip leading aligned models to unsafe action selection at very high rates, directly threatening real-world agent deployments with tool-call logs, replay, or injection. It provides a concrete benchmark (HistoryAnchor-100), broad cross-provider evaluation, strong controls, and an actionable safety red flag that could rapidly influence deployment practices and alignment research. Paper 2 is theoretically ambitious and cross-validates with humans, but its claims may be viewed as less immediately actionable and higher-risk to contest on assumptions.
Paper 2 demonstrates higher potential scientific impact through several factors: (1) It bridges AI and cognitive science with formal impossibility theorems, offering broad interdisciplinary relevance. (2) It provides novel theoretical contributions (three impossibility theorems) with strong empirical validation across both LLMs and pre-registered human experiments. (3) The reframing of cognitive biases as mathematically inevitable consequences of sequential processing is a genuinely novel insight with implications for AI safety, debiasing strategies, and cognitive psychology. (4) The methodological rigor—combining formal proofs, computational experiments across 12 frontier LLMs, and pre-registered human studies—is exceptional. Paper 1, while useful as a methodological guide, is more of a position/review paper without novel empirical or theoretical contributions.
Paper 2 offers fundamental impossibility theorems bridging artificial intelligence and human cognition. By proving mathematically that cognitive biases are inevitable in sequential processing and validating this across LLMs and human studies, it provides a profound theoretical breakthrough. Paper 1 is a solid engineering contribution for optimizing multimodal LLM routing, but lacks the interdisciplinary breadth, theoretical depth, and paradigm-shifting potential of Paper 2.
Paper 1 offers fundamental theoretical contributions by proving impossibility theorems linking AI architecture to cognitive biases, bridging computer science and cognitive psychology. Its rigorous methodology combines mathematical proofs, extensive LLM validation, and pre-registered human experiments. This cross-disciplinary approach provides profound insights into sequential processing limits, granting it significantly broader and deeper scientific impact compared to Paper 2's domain-specific architectural improvements for time-series agents.
Paper 2 has higher likely scientific impact due to stronger novelty (impossibility theorems linking autoregressive constraints to cognitive biases), broad cross-field relevance (ML theory, interpretability, cognitive science, HCI, alignment), and rigorous methodology (formal proofs, multi-LLM validation, preregistered human experiments). Its claims are general and timeless for sequential models and have clear implications for debiasing limits and evaluation. Paper 1 is timely and practically useful for edge/IoT deployment but is primarily an empirical benchmarking study with narrower scope and less foundational theoretical contribution.
Paper 2 has higher potential impact due to stronger novelty (impossibility theorems linking architectural constraints to cognitive biases), broader cross-field relevance (AI theory, cognitive science, human factors, interpretability), and rigorous methodology (formal proofs + validation across 12 LLMs + pre-registered human experiments). Its claims generalize beyond a specific system and offer enduring theoretical constraints and actionable predictions. Paper 1 is timely and practically useful for multi-agent LLM efficiency, but its impact is likely narrower and more incremental, tied to a particular graph-diffusion approach and benchmark performance.
Paper 1 offers stronger scientific impact due to clear theoretical novelty (impossibility theorems linking sequential processing constraints to biases), strong methodological rigor (formal proofs, cross-model validation, preregistered human studies with sizable N and quantitative effects), and broad relevance across ML, cognitive science, and HCI. Its claims are falsifiable and generalizable beyond a specific system. Paper 2 is timely and application-oriented (proactive agents, benchmarks, long-term memory), but the abstract suggests more engineering contribution with less clear theoretical advance and weaker evidence detail, making impact more contingent on adoption.
Paper 2 has broader interdisciplinary impact spanning AI, cognitive science, and psychology. It establishes fundamental impossibility theorems connecting architectural constraints in LLMs to human cognitive biases, offering a unifying theoretical framework validated across both computational (12 LLMs) and human experiments (pre-registered, N=464). This reframing of cognitive biases as mathematically inevitable consequences of sequential processing has profound implications for AI alignment, debiasing strategies, and cognitive science theory. Paper 1, while technically impressive, addresses a narrower problem in physics-based animation with more limited cross-disciplinary reach.
Paper 2 establishes fundamental impossibility theorems regarding sequential processing, drawing profound parallels between AI architectures and human cognitive biases. Its theoretical depth, mathematical rigor, and interdisciplinary scope across computer science and psychology give it a much broader and longer-lasting scientific impact compared to Paper 1, which focuses on a domain-specific applied benchmark for industrial maintenance.
Paper 2 is more likely to have higher scientific impact due to strong novelty (impossibility theorems tying bias to causal masking), breadth (implications for ML theory, LLM alignment/evaluation, and cognitive science), and timeliness given widespread reliance on autoregressive LLMs. It combines formal results with multi-model empirical validation and preregistered human experiments, strengthening methodological rigor and real-world relevance (limits of debiasing, computational tradeoffs). Paper 1 is solid and applied, but advances are more incremental within EEG decoding and likely narrower in cross-field influence.
Paper 2 bridges artificial and biological intelligence through rigorous mathematical proofs, large-scale LLM evaluations, and pre-registered human behavioral studies. Its fundamental theoretical framing of cognitive biases as inevitable constraints of sequential processing offers profound interdisciplinary implications for AI and cognitive science. In contrast, Paper 1 offers a narrower, albeit useful, empirical technique for AI safety.
Paper 2 offers profound theoretical and interdisciplinary contributions by mathematically proving that certain cognitive biases are inevitable in sequential processing. Bridging AI and cognitive science through impossibility theorems, empirical LLM validation, and human experiments provides a fundamental paradigm shift. In contrast, Paper 1 presents a valuable but narrower algorithmic optimization for RL in LLMs. Paper 2's fundamental insights into the nature of intelligence and information processing give it a substantially broader and longer-lasting potential scientific impact across multiple fields.
Paper 1 presents fundamental impossibility theorems proving that certain cognitive biases are mathematically inevitable in sequential processors, bridging AI architecture and cognitive science with formal proofs, empirical validation across 12 LLMs, and pre-registered human experiments. Its theoretical contributions (reframing biases as architectural necessities rather than flaws) have broad implications for AI alignment, cognitive science, and decision-making. Paper 2 introduces a useful text analysis tool but is more incremental—a new NLP pipeline with case studies. Paper 1's formal theoretical framework, methodological rigor, and cross-disciplinary impact give it substantially higher potential.