Rethinking Evaluation of Sparse Autoencoders through the Representation of Polysemous Words

TL;DR

This work argues that evaluating sparse autoencoders (SAEs) solely with reconstruction metrics like $MSE$ and sparsity $L_0$ misses their core goal of extracting monosemantic features for polysemous words. It introduces Poly-Semantic Evaluation (PS-Eval), a model-independent framework built on Word in Context (WiC) to test whether SAE features disentangle polysemantic activations into distinct meanings, using a confusion-matrix based approach. Key findings show that enlarging the SAE latent space improves monosemantic extraction up to a saturation around expand ratio $R\approx 64$, that deeper Transformer layers and the Attention mechanism increase specificity, and that activation-function variants like TopK and JumpReLU do not consistently yield semantic gains. These results advocate for holistic SAE evaluation and offer practical guidance for designing interpretable sparse representations in LLMs and related mechanistic interpretability tasks.

Abstract

Sparse autoencoders (SAEs) have gained a lot of attention as a promising tool to improve the interpretability of large language models (LLMs) by mapping the complex superposition of polysemantic neurons into monosemantic features and composing a sparse dictionary of words. However, traditional performance metrics like Mean Squared Error and L0 sparsity ignore the evaluation of the semantic representational power of SAEs -- whether they can acquire interpretable monosemantic features while preserving the semantic relationship of words. For instance, it is not obvious whether a learned sparse feature could distinguish different meanings in one word. In this paper, we propose a suite of evaluations for SAEs to analyze the quality of monosemantic features by focusing on polysemous words. Our findings reveal that SAEs developed to improve the MSE-L0 Pareto frontier may confuse interpretability, which does not necessarily enhance the extraction of monosemantic features. The analysis of SAEs with polysemous words can also figure out the internal mechanism of LLMs; deeper layers and the Attention module contribute to distinguishing polysemy in a word. Our semantics focused evaluation offers new insights into the polysemy and the existing SAE objective and contributes to the development of more practical SAEs.

Figures (20)

• Figure 1: Evaluation of SAE’s ability to extract monosemantic features from polysemantic activations in LLM. The word "space" in inputs tokens has different meanings depending on the context: context1 refers to "space" as in the universe (e.g., ...astronauts walked in space...), while context2 refers to "space" as in a gap or physical distance (e.g., ...space between his teeth...). SAE is trained by minimizing reconstruction loss and applying a sparsity constraint. The evaluation examines whether the features produced by the SAE can differentiate between the meanings of "space" in different contexts.
• Figure 2: MSE v.s. $\text{L}_{0}$ for expand ratio = 32 across varying $\text{L}_{1}$ coefficients ($\lambda$). The color gradient represents the $\text{L}_1$ coefficient values, with brighter colors indicating larger $\lambda$. As the $\lambda$ increases, the model becomes sparser (lower $\text{L}_{0}$) while MSE gets worse.
• Figure 3: (Left) Cosine distance between LLM activations and SAE features in polysemantic contexts. The histogram compares the distribution of Polysemous Distinction (i.e. 1 - Cosine Similarity) for LLM activations (blue) and SAE features (green), demonstrating that SAE features extract more distinct representations in polysemous contexts. We adopt $R = 128$ and $\text{L}_{1}$ coefficient $\lambda = 0.05$ for the hyperparameters of SAEs. (Right) Comparison of Accuracy vs MSE and $\text{L}_0$ for different Expand Ratios ($\text{L}_1$ Coef = 0.05). The left panel shows the relationship between accuracy and mean squared error (MSE), while the right panel presents accuracy versus normalized $\text{L}_0$ sparsity. The markers represent various expand ratios (8, 16, 32, 64, and 128). Higher expand ratios generally show a trend towards better performance, as indicated by the yellow and green markers corresponding to expand ratios of 64 and 128.
• Figure 4: Comparison of F1 score, Precision, and Recall across different activation functions. The models were trained with JumpReLU, including its STE variant (with varying thresholds), TopK (with different $k$), and ReLU. The results imply that ReLU may be still better than JumpReLU or TopK with the polysemous words.
• Figure 5: Comparison of Specificity across different layers of the SAE in relation to $\text{L}_0$ (left) and MSE (right). Each point represents a layer, from layer 1 to layer 12, with deeper layers (i.e., layer 6 onwards) showing an increasing trend in Specificity. This indicates that deeper layers exhibit more distinct activations for polysemous words, consistent with previous findings that deeper layers in SAEs face greater challenges in training, leading to higher MSE and $\text{L}_0$ scores.