Pre-Intervention Prediction of Sparse Autoencoder Steering Side Effects

Paper 1 addresses a highly timely and critical problem in AI safety and interpretability: controlling the steering side effects of Large Language Models using Sparse Autoencoders. Its empirical breadth across major models (Llama-3, Gemma-2, etc.) and direct applicability to LLM alignment gives it broader immediate relevance and higher potential impact compared to Paper 2, which offers a solid but more specialized theoretical contribution to contextual bandit algorithms.

Paper 2 addresses a critical and timely scientific problem in mechanistic interpretability and AI safety by analyzing and predicting the side effects of SAE feature steering across state-of-the-art language models. This contributes fundamental insights into model representations. In contrast, Paper 1 presents a highly useful engineering and workflow tool for manipulating model weights. While valuable for reproducibility, Paper 2's direct theoretical and empirical contributions to understanding LLM behavior offer higher potential for broad scientific impact.

Paper 2 achieves a rate-optimal result for contextual queueing bandits, improving the regret rate from O(T^{-1/4}) to O(T^{-1/2}) and proving a matching minimax lower bound. This fully characterizes the fundamental complexity of the problem, which is a clean and definitive theoretical contribution with broad implications for operations research, scheduling, and online learning theory. Paper 1 addresses an important practical concern (SAE steering side effects) but its contributions are more empirical and incremental, with findings that are model- and setting-dependent, limiting generalizability. Paper 2's tight theoretical result has more lasting and broadly applicable scientific impact.

Paper 2 addresses a critical cybersecurity problem in power systems with a principled, physically-grounded solution (PCNSA) that has immediate real-world applications in protecting critical infrastructure. It offers a novel theoretical contribution (proving that conventional preprocessing violates subspace alignment) with rigorous validation across standard benchmarks. Paper 1, while technically interesting for the mechanistic interpretability community, addresses a narrower problem (predicting SAE steering side effects) with model-dependent results and limited generalizability, serving primarily the AI safety/interpretability niche rather than broader scientific or engineering communities.

Paper 2 likely has higher scientific impact due to timeliness and broad relevance to mechanistic interpretability and controllability of modern LLMs. It addresses a practical bottleneck—unpredictable side effects of SAE feature steering—by proposing a general pre-intervention screening framework validated across multiple model families and SAE variants, with methodological controls (held-out screening, residualization, dictionary-width comparison). The resulting tooling could improve safety/alignment workflows and steerability across many applications. Paper 1 is novel and useful for geolocalizing time series, but its impact is more domain-specific (e.g., power systems) and less cross-cutting across fields.

Paper 2 addresses a critical bottleneck in mechanistic interpretability—predicting and mitigating side effects in feature steering. Its comprehensive evaluation across multiple state-of-the-art architectures and SAE variants provides foundational insights crucial for AI alignment and safety. While Paper 1 offers a valuable practical compression technique, Paper 2 provides a novel theoretical framework with profound implications for understanding and safely controlling large language models.

Paper 1 presents a foundational advance in representation learning that spans multiple modalities (images, 3D shapes, climate data), offering massive efficiency gains (42x less memory) over existing meta-learning approaches. Its general-purpose nature ensures broad applicability across various scientific and engineering disciplines. In contrast, while Paper 2 is highly relevant to the timely subfield of LLM interpretability and AI safety, its scope is narrower and its findings are highly dependent on specific model and dictionary settings.

Paper 2 has higher likely impact due to broader real-world applicability and cross-domain relevance: it introduces a general, rigorously controlled audit framework for aperiodic (1/f) reliance that affects interpretability and clinical validity across EEG and ECG tasks, architectures, and even foundation models. Its intervention + sham + simulation methodology strengthens causal claims and suggests an actionable standard practice for biomedical ML. Paper 1 is timely and novel for mechanistic interpretability, but its applications are narrower (SAE steering) and findings are more model/dictionary dependent, limiting immediate generalization.

Paper 2 has higher potential impact due to a clearer theoretical contribution with broad relevance: it revises established gradient-flow predictions by proving qualitatively different behavior under practical discrete GD with large step size (edge-of-stability regime). The results connect sharpness, step size, depth, and pathway symmetry—concepts central across deep learning theory and optimization—and may influence how researchers interpret specialization, stability, and representation sharing in overparameterized models. Paper 1 is timely and useful for mechanistic interpretability workflows, but its effects are model/dictionary-dependent and more narrowly scoped to SAE steering practice.

Paper 2 addresses a broadly applicable problem (LLM routing with cost-performance trade-offs) relevant to the rapidly growing LLM deployment ecosystem, with a novel meta-learning formulation that handles heterogeneous user preferences. Its practical applicability is immediate and spans many real-world settings. Paper 1, while technically rigorous and addressing an important interpretability question about SAE steering side effects, is narrower in scope—focused on a specific mechanistic interpretability technique—and its findings are model/dictionary-dependent, limiting generalizability. Paper 2's broader audience and direct practical utility give it higher potential impact.