Trajectory Geometry of Transformer Representations Across Layers
Vishal Pandey, Gopal Singh
Abstract
Understanding how transformer representations evolve across layers, not merely what they encode, remains an open problem in mechanistic interpretability. We recast the transformer forward pass as a discrete population trajectory through a high-dimensional representation manifold, drawing on geometric tools from computational neuroscience. Rather than probing for pre-specified features, we characterize trajectory geometry using five metrics computed directly in the ambient space: trajectory length, curvature, a semantic convergence index, layerwise cosine similarity, and representational stability. Across three model families (GPT-2, TinyLlama, Qwen2.5) and five controlled prompt families, we report four findings. First, semantically related prompts converge significantly in middle-to-late layers (peak CI 0.41--0.58, p<0.001, Mann-Whitney U), consistent with attractor-like dynamics. Second, reasoning tasks produce trajectories of greater curvature than lexical variations (0.71--0.83 rad vs. 0.27--0.31 rad), suggesting curvature encodes computational complexity. Third, ambiguous tokens exhibit trajectory bifurcation with up to 5.6x representational separation by the final layer, absent in unambiguous controls. Fourth, layerwise cosine similarity reveals a universal three-phase structure: encoding, elaboration, and output preparation, consistent across all three architectures. All four effects vanish under shuffled-layer and random-embedding controls. We release a fully open-source, model-agnostic pipeline and argue that trajectory geometry constitutes a principled, probe-free lens for mechanistic interpretability.
AI Impact Assessments
(1 models)Scientific Impact Assessment: Trajectory Geometry of Transformer Representations Across Layers
1. Core Contribution
This paper proposes treating the transformer forward pass as a discrete population trajectory through high-dimensional representation space, borrowing conceptual tools from computational neuroscience (neural population dynamics). The authors define five geometric metrics—trajectory length, curvature, semantic convergence index, layerwise cosine similarity, and representational stability—computed in the full ambient space without dimensionality reduction. They report four findings across three model families (GPT-2, TinyLlama, Qwen2.5): semantic convergence in middle-to-late layers, curvature encoding computational complexity, trajectory bifurcation for ambiguous tokens, and a universal three-phase computational structure.
The conceptual contribution—viewing the layer sequence as a geometric trajectory rather than a collection of independent snapshots—is appealing and provides a unifying lens. However, the individual metrics themselves are relatively straightforward (Euclidean distances, cosine similarities, turning angles), and several have been implicitly or explicitly used in prior work on layer-wise representation analysis, even if not packaged under the "trajectory geometry" umbrella.
2. Methodological Rigor
Strengths in experimental design: The paper includes four control experiments (random labels, random embeddings, shuffled layers, multiple projections) that systematically rule out several confounds. The statistical protocol (Mann-Whitney U, Benjamini-Hochberg correction, bootstrap CIs, Cohen's d) is appropriate and well-documented.
Significant concerns:
3. Potential Impact
The paper's primary value is as a descriptive framework that organizes existing intuitions about layer-wise processing into a geometric vocabulary. The open-source pipeline is a practical contribution that lowers barriers to this type of analysis.
However, the actionable impact is limited by several factors:
The connection to neuroscience population dynamics is interesting conceptually but remains metaphorical—no formal mapping is established, and the authors wisely avoid claiming biological correspondence.
4. Timeliness & Relevance
The paper addresses a genuine need in mechanistic interpretability for methods that characterize global representation dynamics rather than individual components. The probe-free nature is appealing given well-known concerns about probing methodology (probes can find structure that isn't causally relevant). The timing is good given growing interest in representation engineering and understanding how to intervene in model internals.
However, the paper enters a crowded space. CKA analyses, logit lens approaches, and recent work on representation similarity across layers already address overlapping questions. The incremental advance over, say, simply computing CKA across consecutive layers is not always clear.
5. Strengths & Limitations
Key Strengths:
Notable Weaknesses:
6. Additional Observations
The Figure 5 caption mentions "6.5x bifurcation ratio in mock data," which is concerning—it suggests at least some figures may be generated from synthetic data rather than actual experimental results. If confirmed, this would significantly undermine the paper's empirical claims and credibility.
The theoretical bridge to dynamical systems, while suggestive, remains informal. No convergence guarantees, stability analysis, or formal attractor characterization is provided.
Summary
This is a well-organized paper with a clean conceptual framework and appropriate statistical methodology, but it is ultimately a descriptive study with limited novelty in its individual components, a very small evaluation scope, and no causal validation. The primary contribution is packaging existing geometric intuitions into a coherent framework with an accessible pipeline. The "mock data" issue in Figure 5 raises concerns about result integrity that would need to be resolved.
Generated Jun 9, 2026
Comparison History (20)
Paper 1 introduces a fundamental systems-level optimization (fused GPU kernel) for a widely used algorithm (GMMs), mirroring the highly impactful 'FlashAttention' approach. Its concrete improvements—20x speedup and 100x larger dataset capacity—along with immediate applications in approximate nearest-neighbor search, guarantee high real-world utility and widespread adoption across general ML and vector database pipelines. While Paper 2 offers valuable theoretical insights into LLM interpretability, Paper 1's practical, generalizable scalability breakthrough presents a more immediate and measurable scientific and engineering impact.
Paper 2 addresses a high-impact real-world problem (semiconductor manufacturing optimization) with a novel event-driven RL framework that demonstrates significant practical gains in throughput and utilization. Its contributions—scalable deep RL for long-horizon, high-dimensional industrial control with validated results on industry-realistic scenarios—have broader potential impact across manufacturing, operations research, and complex systems engineering. Paper 1 offers interesting geometric analysis of transformer representations but is more descriptive/analytical, with less immediate practical applicability and narrower scope within mechanistic interpretability.
Paper 2 addresses the highly active field of mechanistic interpretability in transformers using innovative geometric tools from computational neuroscience. By analyzing representation manifolds across modern LLMs, it offers a novel, probe-free framework that could broadly impact how we understand LLM reasoning and internal representations. While Paper 1 presents a solid methodological improvement for feature attribution, Paper 2's insights into the fundamental workings of large language models have greater timeliness, broader relevance, and higher potential to shape future research directions.
Paper 2 has higher impact potential due to strong real-world applicability and timeliness: scaling formal verification to multi-GPU directly addresses a major bottleneck for safety-critical deployment. Methodologically, it contributes engineering/algorithmic advances (TP and especially FSDP with bitwise-identical bounds), integrates with complete verification, and demonstrates on competitive benchmarks including a large ResNet case. Its insights about alpha-tensor memory bottlenecks guide future work. Paper 1 is novel and useful for interpretability, but is more descriptive/diagnostic and less likely to unlock immediate, broad operational capabilities than scalable verification.
Paper 2 introduces a novel, probe-free geometric framework bridging computational neuroscience and LLM interpretability. Its discovery of universal trajectory patterns, attractor-like dynamics, and curvature-complexity links offers broader theoretical insights and applications across architectures than Paper 1's targeted, albeit rigorous, methodological critique of attention head clustering.
Paper 1 offers a more methodologically rigorous and broadly applicable contribution to transformer interpretability, a highly active and impactful research area. Its geometric framework is model-agnostic, validated across multiple architectures, and provides concrete, reproducible metrics with appropriate controls. The open-source pipeline enhances practical impact. Paper 2 addresses a fascinating but more niche philosophical-computational question about minimal agency, with results limited to a 192-dimensional GRU in toy environments, making generalization uncertain. Paper 1's timeliness in the booming LLM interpretability field gives it broader immediate relevance and citation potential.
Paper 1 likely has higher impact due to a clearer algorithmic contribution with direct performance gains on a central practical bottleneck: long-horizon credit assignment under sparse rewards in agentic RL/LLM tool-use settings. PBSD introduces a principled Bayesian reweighting that is compatible with standard policy optimization and targets generalization from short-context training to long-context inference—highly timely for real-world agents. Paper 2 offers valuable interpretability metrics and an open pipeline, but its impact may be more descriptive/diagnostic and dependent on downstream adoption, with less immediate capability improvement.
Paper 2 likely has higher impact: it advances a theoretically grounded algorithm for seeded clustering in submodular hypergraphs with provable locality, finite-time convergence/suboptimality guarantees, and robust sweep-cut guarantees—contributions that are broadly reusable across graph/hypergraph learning, optimization, and network science. The “local-at-every-iteration” property addresses a concrete scalability gap with clear real-world applications (large-scale clustering/community detection). Paper 1 is timely and interesting for interpretability, but its metrics-based geometric characterization may be less foundational and generalizable than Paper 2’s algorithmic and theoretical results.
Paper 1 addresses a concrete, well-defined problem (heterogeneous long-term treatment effect estimation) with novel theoretical contributions (Neyman-orthogonality, variance control via overlap weights) and practical applicability in marketing, economics, and medicine. It provides rigorous theoretical guarantees and validates on real-world data. Paper 2 offers interesting geometric characterizations of transformer representations but is primarily descriptive/observational, with findings (e.g., three-phase structure, curvature patterns) that, while informative, are less likely to drive methodological change. Paper 1's contributions are more actionable and methodologically rigorous, with broader applied impact.
Paper 1 introduces a novel geometric framework for understanding transformer representations that is model-agnostic, probe-free, and applicable across architectures. Its findings (attractor dynamics, curvature encoding complexity, trajectory bifurcation, universal three-phase structure) are broadly relevant to the entire mechanistic interpretability community and potentially to all transformer-based AI systems. Paper 2 makes a solid but narrower contribution—adapting tabular foundation models for latent-space Bayesian optimization in molecular design. While useful, it addresses a more specialized problem with incremental methodological innovation (continued pretraining with regularization). Paper 1's breadth of impact across interpretability, neuroscience-inspired AI analysis, and its open-source toolkit give it higher potential impact.