Circuit Insights: Towards Interpretability Beyond Activations

TL;DR

The paper confronts the challenge of scalable mechanistic interpretability for neural circuits in language models by moving beyond activations. It introduces WeightLens, a weight-based interpretability framework, and CircuitLens, a circuit-based analysis approach that uncovers how features arise from component interactions and how they affect outputs. WeightLens provides concise, dataset-free feature descriptions from weights, while CircuitLens clusters circuit patterns to tackle polysemanticity and reveals input-driven triggers and output influences. Together, these methods enhance robustness and scalability of automated interpretability, reducing reliance on external LLM explainer prompts and enabling more faithful, mechanistic insights at scale.

Abstract

The fields of explainable AI and mechanistic interpretability aim to uncover the internal structure of neural networks, with circuit discovery as a central tool for understanding model computations. Existing approaches, however, rely on manual inspection and remain limited to toy tasks. Automated interpretability offers scalability by analyzing isolated features and their activations, but it often misses interactions between features and depends strongly on external LLMs and dataset quality. Transcoders have recently made it possible to separate feature attributions into input-dependent and input-invariant components, providing a foundation for more systematic circuit analysis. Building on this, we propose WeightLens and CircuitLens, two complementary methods that go beyond activation-based analysis. WeightLens interprets features directly from their learned weights, removing the need for explainer models or datasets while matching or exceeding the performance of existing methods on context-independent features. CircuitLens captures how feature activations arise from interactions between components, revealing circuit-level dynamics that activation-only approaches cannot identify. Together, these methods increase interpretability robustness and enhance scalable mechanistic analysis of circuits while maintaining efficiency and quality.

Figures (10)

• Figure 1: Process of layerwise input-invariant analysis via WeightLens.
• Figure 2: Types of attributions in transcoders.
• Figure 3: Percentage of validated feature descriptions per layer obtained via WeightLens.
• Figure 4: Comparison of Neuronpedia feature descriptions for token-based features (i.e., those for which descriptions were successfully generated via WeightLens) and context-based features, whose activations are strongly influenced by context.
• Figure 5: Clusters of activating inputs for L12F619 and their patterns, obtained through attribution to attention heads. Activations are highlighted in green.