On the Theoretical Foundation of Sparse Dictionary Learning in Mechanistic Interpretability
Yiming Tang, Harshvardhan Saini, Yizhen Liao, Dianbo Liu
TL;DR
This work presents the first unified theoretical framework for Sparse Dictionary Learning (SDL) in mechanistic interpretability, treating SAEs, transcoders, and crosscoders as instances of a single optimization problem under the Linear Representation Hypothesis and Superposition Hypothesis. It proves the existence of approximate global minima with reconstruction error scaling as O(ε^2) and derives necessary and sufficient conditions for global optimality under extreme sparsity, effectively decomposing the problem into per-feature constraints. The analysis further links spurious local minima to feature absorption, offering a rigorous explanation for observed phenomena such as dead neurons and neuron resampling, and validates key predictions with controlled experiments. Collectively, the paper provides a principled foundation for understanding when SDL methods recover ground-truth interpretable features and how their optimization landscapes relate to empirical interpretability phenomena.
Abstract
As AI models achieve remarkable capabilities across diverse domains, understanding what representations they learn and how they process information has become increasingly important for both scientific progress and trustworthy deployment. Recent works in mechanistic interpretability have shown that neural networks represent meaningful concepts as directions in their representation spaces and often encode many concepts in superposition. Various sparse dictionary learning (SDL) methods, including sparse autoencoders, transcoders, and crosscoders, address this by training auxiliary models with sparsity constraints to disentangle these superposed concepts into interpretable features. These methods have demonstrated remarkable empirical success but have limited theoretical understanding. Existing theoretical work is limited to sparse autoencoders with tied-weight constraints, leaving the broader family of SDL methods without formal grounding. In this work, we develop the first unified theoretical framework considering SDL as one unified optimization problem. We demonstrate how diverse methods instantiate the theoretical framwork and provide rigorous analysis on the optimization landscape. We provide the first theoretical explanations for some empirically observed phenomena, including feature absorption, dead neurons, and the neuron resampling technique. We further design controlled experiments to validate our theoretical results.