Empirical Evaluation of Progressive Coding for Sparse Autoencoders

TL;DR

This work tackles efficient, interpretable progressive coding for sparse autoencoders in large language models. It proposes two approaches—Matryoshka SAEs (nested, shared-weight models) and dictionary-pruning via permutation ordering—grounded in the Dictionary Power Law hypothesis to enable high-fidelity reconstructions at varying granularity $G$. Empirically, Matryoshka SAEs outperform baselines on granularity-vs-reconstruction and downstream LM loss, while permutation-based methods offer interpretability advantages; power-law structure is observed across model scales. The authors also derive scaling laws relating reconstruction loss to model size $N$, granularity $g$, and sparsity $k$, and discuss practical limitations such as feature-splitting and decoder kernel inefficiencies, proposing directions for improved efficiency and larger-scale validation.

Abstract

Sparse autoencoders (SAEs) \citep{bricken2023monosemanticity,gao2024scalingevaluatingsparseautoencoders} rely on dictionary learning to extract interpretable features from neural networks at scale in an unsupervised manner, with applications to representation engineering and information retrieval. SAEs are, however, computationally expensive \citep{lieberum2024gemmascopeopensparse}, especially when multiple SAEs of different sizes are needed. We show that dictionary importance in vanilla SAEs follows a power law. We compare progressive coding based on subset pruning of SAEs -- to jointly training nested SAEs, or so-called {\em Matryoshka} SAEs \citep{bussmann2024learning,nabeshima2024Matryoshka} -- on a language modeling task. We show Matryoshka SAEs exhibit lower reconstruction loss and recaptured language modeling loss, as well as higher representational similarity. Pruned vanilla SAEs are more interpretable, however. We discuss the origins and implications of this trade-off.

Figures (20)

• Figure 1: Illustrating progressive coding, the dark part highlight the ressources not used by the model at inference time.
• Figure 2: Power law fits for eigenvalues of the covariance matrix, $E[activation^2]$ and activation frequency ($E[\mathbbm{1}{|activation| > 0}]$). We fit a linear regression model to the logarithmically transformed values and display the coefficient and fit for each. We analyze three models of various sizes (65k, 32k, 16k) with consistent sparsity ratios (256-65k, 128-32k, 64-16k).
• Figure 3: An illustration of dictionary permutation with function $\pi$, Both models will produce the same output given the same input
• Figure 4: Granularity vs FVU (normalized reconstruction loss) for non-permuted(baseline), permuted based on $E[\text{activation}^2]$ and $E[\mathbbm{1}\{\text{activation} > 0\}]$. Relative sparsity is fixed such that k non-zero / granularity is constant for all granularities
• Figure 5: Architectural diagram of the Matryoshka SAE, showing nested latent representations of decreasing dimensionality. The encoder and decoder are shared by each nesting