Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Attention Sinks Induce Gradient Sinks

Yihong Chen, Quanming Yao

Abstract

Attention sinks and massive activations are recurring and closely related phenomena in Transformer models. Existing studies have largely focused on the forward pass, making it unclear whether their connection is direct or mediated by a training-time mechanism. We study this question from the perspective of backpropagation. Empirically and theoretically, we show that under causal mask, attention sinks can induce pronounced gradient concentration, which we term gradient sinks. Furthermore, in pre-norm architectures with RMSNorm, massive activations can be understood as an adaptive response to this localized gradient pressure during training. To test this hypothesis, we introduce V-scale, a modification that adjusts value-path backpropagated gradients. In pretrained V-scale models, attention sinks are preserved whereas massive activations are suppressed. These results support the interpretation that gradient sink is a key training-time mediator linking attention sinks and massive activations.

Attention Sinks Induce Gradient Sinks

Abstract

Attention sinks and massive activations are recurring and closely related phenomena in Transformer models. Existing studies have largely focused on the forward pass, making it unclear whether their connection is direct or mediated by a training-time mechanism. We study this question from the perspective of backpropagation. Empirically and theoretically, we show that under causal mask, attention sinks can induce pronounced gradient concentration, which we term gradient sinks. Furthermore, in pre-norm architectures with RMSNorm, massive activations can be understood as an adaptive response to this localized gradient pressure during training. To test this hypothesis, we introduce V-scale, a modification that adjusts value-path backpropagated gradients. In pretrained V-scale models, attention sinks are preserved whereas massive activations are suppressed. These results support the interpretation that gradient sink is a key training-time mediator linking attention sinks and massive activations.
Paper Structure (14 sections, 2 theorems, 6 equations, 5 figures)

This paper contains 14 sections, 2 theorems, 6 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Pre-norm Transformers
  4. Forward metrics for attention sinks and activations
  5. Backward observables
  6. From attention sink to gradient sink
  7. Empirical observations
  8. Theoretical results
  9. Gradient reshaping by massive activations
  10. V-scale: a value-path gradient valve
  11. Definition and design intuition
  12. Experiments
  13. Conclusions
  14. Limitations

Key Result

Theorem 1

Assume $g_t = \mu + \varepsilon_t$, where $\mathbb{E}[\varepsilon_t]=0$, $\mathrm{Tr}(\mathrm{Cov}(\varepsilon_t)) \le \sigma^2$ for all $t$, and $|\mathrm{Tr}(\mathrm{Cov}(\varepsilon_t,\varepsilon_{t'}))| \le \rho$ for all $t \ne t'$. Then we have Hence stronger sink columns imply systematically larger value-path gradient concentration.

Figures (5)

  • Figure 1: Forward phenomena in baseline models. Top row: 0.1B model; bottom row: 0.3B model. From left to right: attention sink mass, thresholded sink rate, residual-stream output norm, and MLP output norm. We compare the first token (token 0) with the mean and maximum over the rest early tokens (positions 1--15). The results confirm the co-occurrence of AS and MA.
  • Figure 2: Token-wise gradient norms of QKV across training checkpoints. Top row: 0.1B model; bottom row: 0.3B model. From left to right: gradient norms of query, key, and value as functions of token position, averaged over layers. In both scales, key and especially value gradients exhibit a pronounced spike at token 0.
  • Figure 3: Scatter plots relating gradient reshaping to input activation norms of Attention block. Top row: 0.1B model; bottom row: 0.3B model. From left to right: $\log_{10}\mathrm{Bloat}^{\mathrm{attn}}$ vs. $\log_{10}\|h_{\mathrm{in}}\|$, $\log_{10}\mathrm{Compress}^{\mathrm{attn}}$ vs. $\log_{10}\|h_{\mathrm{in}}\|$, and $\log_{10}\mathrm{Change}^{\mathrm{attn}}$ vs. $\log_{10}\|h_{\mathrm{in}}\|$. Each point is colored by token group (token 0, token 1, tokens 2--3, tokens 4--7, and the rest early tokens 8--15). Large-activation points of token 0 occupy the high-bloat regime. Compression exhibits an approximately linear inverse relationship with activation scale. The net residual-level change remains concentrated near $\log_{10}1=0$ except for a small set of first-layer points.
  • Figure 4: Schematic of V-scale inside a pre-norm Transformer block.
  • Figure 5: Forward phenomena in baseline and V-scale models. Top row: 0.1B models; bottom row: 0.3B models. From left to right: thresholded sink rate, residual-stream output norm, MLP output norm, and Attention output norm. We compare the first token with the mean over the rest early tokens (positions 1--15). Across both scales, V-scale largely preserves sink behavior while reducing the activation norms of token 0, with the larger reduction appearing in MLP outputs.

Theorems & Definitions (3)

  • Theorem 1: V-side gradient control by sink statistics
  • Theorem 2: Activation-dependent compression under RMSNorm
  • Remark 1