Interpretability as Compression: Reconsidering SAE Explanations of Neural Activations with MDL-SAEs

TL;DR

This work reframes Sparse Autoencoders as mechanisms for communicating explanations of neural activations, using Minimal Description Length to balance accuracy and conciseness. It introduces independent additivity as a key requirement for human-interpretable features and demonstrates that MDL-driven SAEs yield more composable, stroke-like features on MNIST than naively sparse or dense alternatives. By analyzing feature-splitting phenomena and hierarchical architectures, the paper argues that MDL-guided design can reduce noninterpretable fragmentation and enable more efficient, structured representations. The study connects interpretability with information-theoretic principles such as rate–distortion and compression, offering a principled path for evaluating and designing interpretable AI systems and outlining avenues for future work to integrate entropy-based objectives into training.

Abstract

Sparse Autoencoders (SAEs) have emerged as a useful tool for interpreting the internal representations of neural networks. However, naively optimising SAEs for reconstruction loss and sparsity results in a preference for SAEs that are extremely wide and sparse. We present an information-theoretic framework for interpreting SAEs as lossy compression algorithms for communicating explanations of neural activations. We appeal to the Minimal Description Length (MDL) principle to motivate explanations of activations which are both accurate and concise. We further argue that interpretable SAEs require an additional property, "independent additivity": features should be able to be understood separately. We demonstrate an example of applying our MDL-inspired framework by training SAEs on MNIST handwritten digits and find that SAE features representing significant line segments are optimal, as opposed to SAEs with features for memorised digits from the dataset or small digit fragments. We argue that using MDL rather than sparsity may avoid potential pitfalls with naively maximising sparsity such as undesirable feature splitting and that this framework naturally suggests new hierarchical SAE architectures which provide more concise explanations.

Figures (5)

• Figure 1: A schematic showing a sparse autoencoder (SAE) being used to communicate an input by transmitting the encoded activations and decoding them into a reconstruction of the input.
• Figure 2: Examples of different SAE architectures. All but nonlinear decoders are compatible with independent additivity as feature activations correspond to adding a separate vector to the output. Architectures with directed tree decoders or which allow for vectors lying within a subspace are potentially more communication efficient since a child node can only be active if its parent node is active.
• Figure 3: Finding the minimal description length (MDL) solution for SAEs trained on MNIST. A) Description length vs sparsity ($L_0$) for a set of hyperparameters with the same reconstruction error. B) Plot of the number of alive features as a function of sparsity ($L_0$). C) A random sample of SAE features at the 95th, 80th, 50th, 20th, and 5th percentiles of feature density respectively.
• Figure 4: A toy model of undesirable feature splitting. The SAE can learn two boolean features without feature splitting (A) or three mutually exclusive boolean features with feature splitting (B) which always has lower $L_0$. Minimizing description length provides a decision boundary (C) for when feature splitting is preferred or not.
• Figure 5: Two naturally hierarchical boolean features, such as "Animal" and "Bird", can be learned as separate mutually exclusive features (A) or in hierarchy (B) where the child feature can only be active if the parent feature is active, captured by the conditional probability $p_{B|A}$. C) The hierarchical case always has lower description length (DL) since the child feature's activations need not be sent when the parent is not active.

Theorems & Definitions (4)

• Definition 2.1
• Definition 2.2
• Definition 2.3
• Definition 3.1