Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the geometry and topology of representations: the manifolds of modular addition

Gabriela Moisescu-Pareja, Gavin McCracken, Harley Wiltzer, Vincent Létourneau, Colin Daniels, Doina Precup, Jonathan Love

TL;DR

This paper shows that neural networks trained on the modular addition task $(a+b) \bmod n$ across architectures (MLP-Add, MLP-Concat, and uniform/trainable attention) learn representations that are topologically and geometrically equivalent. By modeling first-layer preactivations as combinations of simple sinusoidal components and invoking symmetry, the authors prove that the learned manifolds are either a vector-addition disc or a torus, with the disc arising as a projection of the torus. They empirically validate these predictions using phase distributions (Phase Alignment Distributions) and topological signatures (Betti numbers) across hundreds of networks, demonstrating that Clock and Pizza are not fundamentally distinct circuits but different factorizations of the same torus-to-circle map. The approach provides a scalable, quantitative toolkit (PADs and persistent homology) to compare learned representations and supports the universality hypothesis for modular addition, potentially informing interpretation and generalization to broader neural computation tasks.

Abstract

The Clock and Pizza interpretations, associated with architectures differing in either uniform or learnable attention, were introduced to argue that different architectural designs can yield distinct circuits for modular addition. In this work, we show that this is not the case, and that both uniform attention and trainable attention architectures implement the same algorithm via topologically and geometrically equivalent representations. Our methodology goes beyond the interpretation of individual neurons and weights. Instead, we identify all of the neurons corresponding to each learned representation and then study the collective group of neurons as one entity. This method reveals that each learned representation is a manifold that we can study utilizing tools from topology. Based on this insight, we can statistically analyze the learned representations across hundreds of circuits to demonstrate the similarity between learned modular addition circuits that arise naturally from common deep learning paradigms.

On the geometry and topology of representations: the manifolds of modular addition

TL;DR

This paper shows that neural networks trained on the modular addition task across architectures (MLP-Add, MLP-Concat, and uniform/trainable attention) learn representations that are topologically and geometrically equivalent. By modeling first-layer preactivations as combinations of simple sinusoidal components and invoking symmetry, the authors prove that the learned manifolds are either a vector-addition disc or a torus, with the disc arising as a projection of the torus. They empirically validate these predictions using phase distributions (Phase Alignment Distributions) and topological signatures (Betti numbers) across hundreds of networks, demonstrating that Clock and Pizza are not fundamentally distinct circuits but different factorizations of the same torus-to-circle map. The approach provides a scalable, quantitative toolkit (PADs and persistent homology) to compare learned representations and supports the universality hypothesis for modular addition, potentially informing interpretation and generalization to broader neural computation tasks.

Abstract

The Clock and Pizza interpretations, associated with architectures differing in either uniform or learnable attention, were introduced to argue that different architectural designs can yield distinct circuits for modular addition. In this work, we show that this is not the case, and that both uniform attention and trainable attention architectures implement the same algorithm via topologically and geometrically equivalent representations. Our methodology goes beyond the interpretation of individual neurons and weights. Instead, we identify all of the neurons corresponding to each learned representation and then study the collective group of neurons as one entity. This method reveals that each learned representation is a manifold that we can study utilizing tools from topology. Based on this insight, we can statistically analyze the learned representations across hundreds of circuits to demonstrate the similarity between learned modular addition circuits that arise naturally from common deep learning paradigms.
Paper Structure (39 sections, 1 theorem, 42 equations, 8 figures, 4 tables)

This paper contains 39 sections, 1 theorem, 42 equations, 8 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Setup and background
  4. Previous interpretations of modular addition in neural networks
  5. Topological Data Analysis
  6. Canonical Manifolds
  7. Simple neuron phase distribution dictates representation manifold
  8. Qualitative analysis of intermediate representations
  9. Principal component analysis (PCA).
  10. Distribution of post-ReLU activations.
  11. Linking PCA geometry and activation distributions.
  12. Methodology
  13. Phase Alignment Distributions.
  14. Betti number distribution.
  15. Results
  16. ...and 24 more sections

Key Result

Theorem 4.1

Let $f\in\mathbf{Z}_p$ for $p\geq 3$, and consider the frequency cluster at layer 1. Let $m$ denote the number of neurons in this cluster, and assume $m\geq 2$. Define the matrix $X\in\mathbb{R}^{p^2\times m}$ according to $X_{(a, b), i} = \cos(\theta_a + \phi^L_i) + \cos(\theta_b + \phi^R_i)$, deno

Figures (8)

  • Figure 1: Clock and Pizza's analytical forms visualized, with frequency assumed to be $f=1$ for simplicity. Each point corresponds to a pair $(a,b)$ after being transformed by the corresponding analytical form and is colored by its sum $(a+b)\mod 59$.
  • Figure 2: PCA of neuron pre-activations for a single frequency cluster across architectures: MLP-Add ($f=27$), Pizza ($f=17$), Clock ($f=21$), MLP-Concat ($f=22$). Each point is an input $(a,b)$, colored by $(d\cdot a + d\cdot b)\bmod 59$ corresponding to the network's output (see Section \ref{['sec:prev-interps']}). Pizza and Clock are nearly identical to each other and to MLP-Add, but differ strongly from MLP-Concat. Note that the trained pizza and clock models being PCA'd are downloaded directly from zhong2023the's Github: model_p99zdpze5l.pt and model_l8k1hzciux.pt.
  • Figure 3: MLP-Add(f=27), Pizza(f=17), Clock(f=21), MLP-Concat(f=22). Normalized sum of post-activations in clusters in each architecture over all $(a,b)$. Clusters in MLP vector add, Attention 0.0 and 1.0 activate strongest on $(a,b)$ with $a$ close to $b$: the activation strength decreases with distance from $a=b$. Clusters in MLP-Concat activate almost equivariantly.
  • Figure 4: Different factorizations of the torus-to-circle map. We find first-layer intermediate representations to be either a torus or a disc (resembling vector addition on the circle). Later layers can construct a circle, and the logits approximate a circle. See Appendix \ref{['app:hypothesis']} regarding the torus-to-circle factorization.
  • Figure 5: Log-density heatmaps for the distribution of neuron maximum activations (top) and activation center of mass (bottom) across 703 trained models. Attention 0.0 and 1.0 architectures exhibit modest off-diagonal spread compared to MLP-Add, but remain constrained by architectural bias toward diagonal alignment. The maximum mean discrepancy scores between Attention 0.0 and 1.0 are 0.0237 and 0.0181 in rows 1 and 2 respectively, indicating they are very similar distributions.
  • ...and 3 more figures

Theorems & Definitions (4)

  • Definition 3.1: Simple Neurons
  • Theorem 4.1
  • Remark 4.2
  • proof