Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spectral Superposition: A Theory of Feature Geometry

Georgi Ivanov, Narmeen Oozeer, Shivam Raval, Tasana Pejovic, Shriyash Upadhyay, Amir Abdullah

TL;DR

The paper addresses polysemanticity and feature interference in neural networks by shifting from sparse, pairwise feature decompositions to a global, geometry-focused view grounded in spectral theory. It introduces the frame operator $F=WW^{\top}$ and the Gram matrix $M=W^{\top}W$ to connect activation-space geometry with weight-space structure, formalizing per-feature spectral measures $\mu_i$ that describe how each feature distributes its mass across eigen-mpaces. In toy models of superposition, capacity saturation drives spectral localization: features collapse to single eigenspaces, form tight frames within those subspaces, and can be classified geometrically via association schemes; gradient flow dynamics explain the emergence and stability of these configurations. The framework provides a general, basis-invariant interpretability toolset that can diagnose and classify feature geometry beyond toy settings, linking operator theory with practical analysis of neural representations.

Abstract

Neural networks represent more features than they have dimensions via superposition, forcing features to share representational space. Current methods decompose activations into sparse linear features but discard geometric structure. We develop a theory for studying the geometric structre of features by analyzing the spectra (eigenvalues, eigenspaces, etc.) of weight derived matrices. In particular, we introduce the frame operator $F = WW^\top$, which gives us a spectral measure that describes how each feature allocates norm across eigenspaces. While previous tools could describe the pairwise interactions between features, spectral methods capture the global geometry (``how do all features interact?''). In toy models of superposition, we use this theory to prove that capacity saturation forces spectral localization: features collapse onto single eigenspaces, organize into tight frames, and admit discrete classification via association schemes, classifying all geometries from prior work (simplices, polygons, antiprisms). The spectral measure formalism applies to arbitrary weight matrices, enabling diagnosis of feature localization beyond toy settings. These results point toward a broader program: applying operator theory to interpretability.

Spectral Superposition: A Theory of Feature Geometry

TL;DR

The paper addresses polysemanticity and feature interference in neural networks by shifting from sparse, pairwise feature decompositions to a global, geometry-focused view grounded in spectral theory. It introduces the frame operator and the Gram matrix to connect activation-space geometry with weight-space structure, formalizing per-feature spectral measures that describe how each feature distributes its mass across eigen-mpaces. In toy models of superposition, capacity saturation drives spectral localization: features collapse to single eigenspaces, form tight frames within those subspaces, and can be classified geometrically via association schemes; gradient flow dynamics explain the emergence and stability of these configurations. The framework provides a general, basis-invariant interpretability toolset that can diagnose and classify feature geometry beyond toy settings, linking operator theory with practical analysis of neural representations.

Abstract

Neural networks represent more features than they have dimensions via superposition, forcing features to share representational space. Current methods decompose activations into sparse linear features but discard geometric structure. We develop a theory for studying the geometric structre of features by analyzing the spectra (eigenvalues, eigenspaces, etc.) of weight derived matrices. In particular, we introduce the frame operator , which gives us a spectral measure that describes how each feature allocates norm across eigenspaces. While previous tools could describe the pairwise interactions between features, spectral methods capture the global geometry (``how do all features interact?''). In toy models of superposition, we use this theory to prove that capacity saturation forces spectral localization: features collapse onto single eigenspaces, organize into tight frames, and admit discrete classification via association schemes, classifying all geometries from prior work (simplices, polygons, antiprisms). The spectral measure formalism applies to arbitrary weight matrices, enabling diagnosis of feature localization beyond toy settings. These results point toward a broader program: applying operator theory to interpretability.
Paper Structure (29 sections, 51 theorems, 213 equations, 39 figures)

This paper contains 29 sections, 51 theorems, 213 equations, 39 figures.

Table of Contents

  1. Introduction
  2. Motivating Spectral Methods
  3. Generalizable Tools
  4. Solving Toy Models of Superposition
  5. Capacity Saturation is Spectral Measure Localization
  6. Limitations and Conclusion
  7. Impact statement
  8. Feature Geometry
  9. Orbit Algebra Derivation
  10. Association Schemes
  11. Algebra Classification
  12. Simplex algebra
  13. Tight Frame
  14. Spectral Theory
  15. Perturbation Theory
  16. ...and 14 more sections

Key Result

Theorem 1

Assume the model saturates the fractional dimensionality capacity bound, i.e. $\sum_{i=1}^n D_i = \text{rank}(W) = m$. Then, for every feature $i$, the spectral measure collapses to a single Dirac mass $\mu_i = \delta_{\lambda_k}$, centered at some eigenvalue $\lambda_k>0$. This is equivalent to $FW

Figures (39)

  • Figure 1: A diagram showing the concepts of "rock(R)/paper(P)/scissors(S)" and "heads(H)/tails(T)" embedded in a three dimensional space. This is an example of structural interference, where a model causes concepts to interfere because they are related (the hand-game features interfere and the coin features interfere based on the relationships between the objects in each set, but the two sets do not interfere with each other). Current methods, which only extract features, will create a basis with 5 vectors. We capture the relationships between features in activation space using the spectral measure to bin features by the eigenspaces they occupy. When each feature is spectrally localized (occupies a single eigenspace), we can recover the full geometry by covering it with an additional spectral signature. These tools from operator theory let us explore the global character of interference: not just how pairs of features interact, but how all features relate.
  • Figure 2: The games of chance example in more detail, both geometrically and as a matrix. Here, $W_\triangle = [W_1, W_2, W_3]$ represent the feature concepts for "rock", "paper", and "scissors", respectively, arranged as an equilateral triangle in the $xy$-plane. Similarly, $W_D= [W_4, W_5]$ correspond to "heads" and "tails" and form an antipodal pair along the $z$-axis.
  • Figure 3: The elements of $D_3$ in permutation notation.
  • Figure 4: Association Scheme Strata
  • Figure 5: Capacity Saturation across 3,200 runs
  • ...and 34 more figures

Theorems & Definitions (89)

  • Theorem 1: Spectral Localization
  • proof
  • Corollary 2: Projective Linearity
  • Theorem 3: Decomposition into Tight Frames
  • Theorem 4: Spectral Identification of Geometry
  • Corollary 5: Simplex Identification
  • Lemma 1.1: Orthogonality of Permutation Matrix
  • proof
  • Lemma 1.2: Centralizer Membership
  • proof
  • ...and 79 more