Dense SAE Latents Are Features, Not Bugs

TL;DR

Dense SAE latents are shown not to be mere training artifacts but reflect intrinsically dense directions in language-model residual space. Through ablation, antipodality analyses, and a taxonomy across layers, the work demonstrates that dense latents encode position, context binding, entropy regulation, alphabetic token structure, and POS-related signals, with a clear evolution from early structural cues to late output-oriented signals. Steering experiments and entropy analyses reveal functional roles for these latents, including context-dependent concept binding and RMSNorm-related entropy modulation. These findings argue for rethinking sparsity-driven interpretability, suggesting SAE designs that can leverage dense subspaces to capture core model mechanics rather than suppress them, with implications for feature-extraction methods in large language models.

Abstract

Sparse autoencoders (SAEs) are designed to extract interpretable features from language models by enforcing a sparsity constraint. Ideally, training an SAE would yield latents that are both sparse and semantically meaningful. However, many SAE latents activate frequently (i.e., are \emph{dense}), raising concerns that they may be undesirable artifacts of the training procedure. In this work, we systematically investigate the geometry, function, and origin of dense latents and show that they are not only persistent but often reflect meaningful model representations. We first demonstrate that dense latents tend to form antipodal pairs that reconstruct specific directions in the residual stream, and that ablating their subspace suppresses the emergence of new dense features in retrained SAEs -- suggesting that high density features are an intrinsic property of the residual space. We then introduce a taxonomy of dense latents, identifying classes tied to position tracking, context binding, entropy regulation, letter-specific output signals, part-of-speech, and principal component reconstruction. Finally, we analyze how these features evolve across layers, revealing a shift from structural features in early layers, to semantic features in mid layers, and finally to output-oriented signals in the last layers of the model. Our findings indicate that dense latents serve functional roles in language model computation and should not be dismissed as training noise.

Figures (22)

• Figure 1: General Properties of Dense SAE Latents. (a) Ablating the dense-latent subspace (teal) reduces high-density latents compared to the original (blue) and sparse-latent ablations (orange). (b) Encoder cosine similarity between the top 50 latents with highest density. (c) Dense latents exhibit high antipodality score: they form pairs that reconstruct specific residual stream directions.
• Figure 2: AbsoluteTopK SAEs show no antipodality. Allowing the SAE to have both positive and negative latent activations removes antipodal dense latents.
• Figure 3: An overview of our taxonomy of dense latents, for every layer. See \ref{['app:class']} for how we created this plot.
• Figure 4: Context-Binding Latents. Activation patterns of layer 12 antipodal pair 7541 (blue, feature 1) and 2009 (red, feature 2). In the first context, they seem to be tracking "casino facts" vs "looking for a buyer", while in the second context, they seem to be tracking "healthcare" vs "press conference". Their corresponding completions are in line with the concepts they activated on.
• Figure 5: Fraction of correct flips when steering, for all latent pairs that have at least one latent $f>0.2$, and $\geq40$ flips. Points are sized by number of flips.