Representation Learning on a Random Lattice
Aryeh Brill
TL;DR
This work treats learned representations as coordinates on a distributional geometry shaped by a target function. It introduces a random lattice model for data distributions and analyzes it with percolation theory, deriving a critical threshold $p_c=\frac{1}{z-1}$ and fractal cluster properties (e.g., $D=4$, $\tau=\tfrac{5}{2}$) in high dimensions. From this geometry, it posits three natural feature categories—context, component, and surface—whose emergence follows a hierarchical, fractal structure, with sparsity and composition depending on dataset regime. The framework offers explanations for observed phenomena in mechanistic interpretability and suggests directions for interpretability methods, including improved sparse dictionary learning and nested latent architectures. Overall, the approach connects data-distribution geometry to practical feature discovery and interpretability in large models.
Abstract
Decomposing a deep neural network's learned representations into interpretable features could greatly enhance its safety and reliability. To better understand features, we adopt a geometric perspective, viewing them as a learned coordinate system for mapping an embedded data distribution. We motivate a model of a generic data distribution as a random lattice and analyze its properties using percolation theory. Learned features are categorized into context, component, and surface features. The model is qualitatively consistent with recent findings in mechanistic interpretability and suggests directions for future research.