Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEs

TL;DR

This paper tackles the problem of cross-run instability in Sparse Autoencoders (SAEs) used for mechanistic interpretability (MI) by arguing that feature consistency should be a central evaluation criterion. It defines Strong Feature Consistency and operationalizes it with the Pairwise Dictionary Mean Correlation Coefficient (PW-MCC), relating it to identifiability through the spark condition and a round-trip property. The authors theoretically justify when TopK SAEs yield consistent dictionaries and validate this with synthetic experiments and a model-organization study, then demonstrate real-world evidence from large language model activations where TopK SAEs achieve high PW-MCC (~0.80) and exhibit frequency-dependent consistency that correlates with semantic explanations. The work advocates a community-wide shift to routinely measure feature consistency, proposes benchmarks, and outlines directions for improving consistency and interpretability in MI for complex models.

Abstract

Sparse Autoencoders (SAEs) are a prominent tool in mechanistic interpretability (MI) for decomposing neural network activations into interpretable features. However, the aspiration to identify a canonical set of features is challenged by the observed inconsistency of learned SAE features across different training runs, undermining the reliability and efficiency of MI research. This position paper argues that mechanistic interpretability should prioritize feature consistency in SAEs -- the reliable convergence to equivalent feature sets across independent runs. We propose using the Pairwise Dictionary Mean Correlation Coefficient (PW-MCC) as a practical metric to operationalize consistency and demonstrate that high levels are achievable (0.80 for TopK SAEs on LLM activations) with appropriate architectural choices. Our contributions include detailing the benefits of prioritizing consistency; providing theoretical grounding and synthetic validation using a model organism, which verifies PW-MCC as a reliable proxy for ground-truth recovery; and extending these findings to real-world LLM data, where high feature consistency strongly correlates with the semantic similarity of learned feature explanations. We call for a community-wide shift towards systematically measuring feature consistency to foster robust cumulative progress in MI.

Figures (44)

• Figure 1: TopK SAE is significantly better than Standard SAE (0.97 vs 0.63) in terms of GT-MCC.
• Figure 2: GT-MCC and PW-MCC for TopK and Standard SAE. PW-MCC follows the same trend as GT-MCC, both converging to comparable values. Shaded region represents max-min range across seeds.
• Figure 3: Left: Redundant regime with high GT-MCC but lower PW-MCC due to selection ambiguity. Right: Compressive regime with lower GT-MCC and PW-MCC. Max-min range across 5 seeds is shaded.
• Figure 4: Token frequency in 1M tokens from Pile, showing the Zipfian distribution in real data, with a long and sparse tail.
• Figure 5: Min activation frequency between matched feature pairs vs. pairwise similarity. Data from two-phase Zipfian model ($d_{\text{gt}}=5000$, $d_{\text{sae}}=1000$). Feature-level similarity captures the influence of local consistency regimes across the frequency spectrum.

Theorems & Definitions (10)

• Definition 1: Spark condition
• Theorem 1: Adapted from hillar2015when
• Definition 2: $\mathcal{T}$-Feature Consistency
• Definition 3: Strong Feature Consistency
• Lemma 1: Two-Vector Decomposition
• proof
• Theorem 2: Round-Trip Implies $k$-Injectivity
• proof
• Corollary 1: Spark Condition for TopK SAEs
• proof