Kurate.org
Rankings
Back to Rankings

Strength Controls Change How Interventions Compare Across Sparse Autoencoder Codes

Jinchen Zhang, Zheyuan Liu, Yihan Zhu, Meng Jiang

Oct 19, 2026arXiv:2610.26489v1
cs.LGcs.CL
Share
Scorecard· 16/16
4.0/10 impact

Carefully executed and honest methodological study, but the narrow scope, partly circular headline result (Q1 uses target outputs), and a niche audience limit expected impact.

Abstract

Changing sparse autoencoder (SAE) codes can alter both intervention direction and magnitude, so comparing intervention effects requires a strength control specifying what is held equal. Norm matching and output calibration provide different controls, but whether interventions yield different effects across codes after strength matching remains unclear. We compare intervention effects under these controls for fixed feature groups in Gemma and Llama and observe: (1) Whole-patch scaling. With one coefficient per input, full-output Kullback-Leibler (KL) calibration leaves smaller mean effect gaps from the encoder-based reference intervention than total-norm matching for nonnegative least-squares (NNLS) refits. On new Llama inputs, total-norm matching loses 11 reference transfers, whereas full-output calibration preserves every transfer decision. (2) Positionwise scaling. On a small matched gender-agreement subset, positionwise norm matching leaves smaller observed mean gaps than full-output calibration. (3) Answer exclusion. Compared with unscaled NNLS, answer-excluded calibration reduces gaps in the measured subject-verb agreement comparisons but increases them for gender refits. Full-output KL calibration also reduces larger gaps for norm-matched constructed candidates, although matching coverage is incomplete in several settings. Since calibration uses the evaluated input's outputs, it serves as a diagnostic of residual disagreement.

AI Impact Assessments

(1 model)

Core Contribution

The paper asks a narrow methodological question in SAE-based interventions: when different sparse codes (the released encoder, NNLS refits, constructed norm-constrained candidates) are used to build a patch for a fixed feature group, how much of the difference in effect reflects a difference in patch *strength* rather than in the code itself? It compares strength controls: total-norm matching, positionwise norm matching, full-output KL calibration (Q1), and answer-excluded KL calibration (Q2). The reference is the encoder-based intervention E, explicitly not treated as ground truth.

Main findings:

  • With one coefficient per input, Q1 leaves smaller NNLS gaps than total-norm matching in all five cohorts. A prospective Llama-128K confirmation (304 inputs) gives a 0.181 vs 0.021 logit gap, and total-norm matching loses 11 of E's 175 transfers while Q1 loses none.
  • Positionwise norm matching can beat Q1 (on 24 gender inputs, descriptively).
  • Answer exclusion helps on subject-verb agreement (SVA) but worsens gender NNLS gaps.
  • Q1 shrinks large gaps for constructed candidates, but coverage is incomplete.

    • The practical message is that the choice of strength control changes conclusions about transfer decisions, and that matched-subset and all-attempt results should be reported together.

    Methodological Rigor

    The paper is unusually careful about its epistemic status.

  • It separates prespecified, prospective, and post hoc analyses (Table 18).
  • It runs a prospective confirmation on new material, with a power calculation, hash-ordered noun selection, and block bootstrap.
  • It retains failed matching attempts and reports coverage failures.
  • It does numerical replay and ships a CPU supplement.
  • Its claims are heavily hedged, and the "diagnostic, not method" framing of Q1 is honest.

    • Weaknesses:

  • The central finding is partly circular. Q1 uses the evaluated input's own outputs to match KL to E, so it is likely to reduce effect gaps from E almost by construction, and the paper acknowledges this. The comparison confounds criterion, information, and compute.
  • The reference E is itself not a validated standard.
  • The scope is small: two templated phenomena, five configurations, one layer per model, and a single base model for the Llama results.
  • Adaptive selection of settings, with many stopped searches in the appendices, limits generalization.
  • The primary 9B interval missed its planned precision.
  • Many comparisons are descriptive with no interval (the gender positionwise subset of 24 inputs).
  • Constructed-candidate coverage fails in most settings.

    • Potential Impact

    The audience is narrow: researchers doing SAE feature interventions, comparing sparse inference methods (amortization gap, identifiability), or doing steering and patching with norm-matched controls. The takeaways are cautionary: norm matching is not enough, output-matching can change transfer decisions, and coverage and all-attempt reporting matter. They are modestly useful as methodological hygiene. The paper provides no new method that practitioners would adopt, because Q1 requires the evaluated outputs. It is unlikely to become a widely reused primitive. Interpretability evaluation sits in a fast-moving area, and the specific numbers are tied to a small set of tasks.

    Timeliness

    The topic is timely. It touches on SAE evaluation, concerns about whether sparse codes are unique, and rigorous intervention comparison. It engages recent work (O'Neill et al., Carbo & Nalisnick, SteerCheck). However, it is a niche corner of that literature, and a heavily qualified one.

    Strengths

  • Honest accounting of evidence status.
  • A prospective confirmation.
  • Transfer-decision analysis showing that small mean gaps can hide flips.
  • The KL decomposition explaining the gender reversal, though it is only descriptive.
  • Reproducibility infrastructure, with pinned revisions and a checker.

    • Limitations

  • The main result is nearly tautological, since calibration uses the target's own outputs.
  • The tasks are narrow and templated, and the effects are tiny relative to encoder effects (median encoder effect about 3–4 logits against gaps of 0.1).
  • The paper is dense and hard to read, with an extremely long appendix, many qualified claims, and many acronyms (Q1/Q2, E, NNLS, SVA, the matched/unmatched variants).
  • Interpretive takeaways are weak: the paper itself says agreement with E is not faithfulness.
  • It offers no actionable recommendation beyond reporting practices.

    • Overall this is a rigorous but low-ceiling, incremental methodology paper. It is likely to be cited modestly within SAE evaluation work.

    Rating:4/ 10
    Significance 3.5Rigor 6.5Novelty 3.5Clarity 4

    Generated Oct 20, 2026

    Comparison History (0)

    No comparisons yet.