The emergence of clusters in self-attention dynamics

TL;DR

The paper develops a rigorous dynamical-systems view of Transformer self-attention by treating tokens as interacting particles with fixed weight matrices. It proves that, depending on the spectrum of the value matrix $V$, tokens asymptotically cluster into distinct geometric objects: in 1D the self-attention matrix converges to a low-rank Boolean form, in the $V=I_d$ case tokens approach the vertices of a shrinking convex polytope, and with a simple positive leading eigenvalue tokens align with up to three hyperplanes. When the leading eigenvalue has multiplicity, clustering combines polytope and subspace behavior, generalizing to richer mixed geometries. The work establishes well-posedness, ties the dynamics to a continuity equation, and supports the theory with numerical experiments on ALBERT-valued matrices. Overall, the results provide a principled explanation for the emergence of leaders and structured representations in self-attention dynamics, with implications for understanding Transformer interpretability and efficiency.

Abstract

Viewing Transformers as interacting particle systems, we describe the geometry of learned representations when the weights are not time dependent. We show that particles, representing tokens, tend to cluster toward particular limiting objects as time tends to infinity. Cluster locations are determined by the initial tokens, confirming context-awareness of representations learned by Transformers. Using techniques from dynamical systems and partial differential equations, we show that the type of limiting object that emerges depends on the spectrum of the value matrix. Additionally, in the one-dimensional case we prove that the self-attention matrix converges to a low-rank Boolean matrix. The combination of these results mathematically confirms the empirical observation made by Vaswani et al. [VSP'17] that leaders appear in a sequence of tokens when processed by Transformers.

Figures (18)

• Figure 1: For $V=I_3$ tokens cluster toward the vertices of a convex polytope (Theorem \ref{['t:Idcase11int']}).
• Figure 2: Elements in $\mathscr{P}$, where $P_{\sigma_i}\in\mathbb{R}^{n\times n}$ are some permutation matrices, and asterisks denote arbitrary non-negative reals which add to $1$.
• Figure 3: An illustration of the asymptotics of $P(t)$ entailed by Theorem \ref{['t:boolean']} for $n=40$ tokens, with $Q=K=1$ and $V=1$. (See \ref{['s: pictures']} for details on computing.) Increasing $n$ has no effect on this behavior of $P(t)$---see Fig. \ref{['f:largenP']}.
• Figure 4: The clouds $\{Kx_i(t)\}_{i\in[20]}$ (green) and $\{Qx_j(t)\}_{j\in[20]}$ (purple) for $d=2$ where pairwise points of clouds are connected by a line of width equal to $P_{ij}(t)$. Here $V\succ0$ and $Q\succ0$ are random matrices and $K=I_2$. The creation of clusters is reflected by the rank $\leq2$ structure of the self-attention matrix $P(t)$. This interaction echoes findings illustrated in the original paper vaswani2017attention---for instance, Figures 3-5 therein.
• Figure 5: A toy example illustrating Theorem \ref{['t:Idcase11int']} with $n=40$ tokens in $\mathbb{R}^3$. Here $Q=K=I_3$. The tokens converge to one of the vertices (leaders) of the limiting convex polytope.

Theorems & Definitions (69)

• Remark 1.1: Discrete time
• Theorem 2.1: Self-attention matrix converges to a low-rank Boolean matrix
• Theorem 3.1: Convergence to points on the boundary of a convex polytope
• Remark 3.2
• Remark 3.3: Rate of convergence
• Remark 3.4: Discrete time
• Definition 4.1
• Theorem 4.2: Convergence toward $\leq3$ hyperplanes
• Conjecture 4.3: Codimension conjecture
• Definition 5.1