A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Training

TL;DR

This work argues that outliers in large language models—attention sinks and residual sinks—are not mere anomalies but function as rescale factors that interact with normalization ($softmax$ and $RMSNorm$) to modulate non-outlier components. The authors propose the outlier-driven rescaling hypothesis and validate it across dense and MoE architectures with token budgets up to $10^{12}$, showing that removing normalization or simply clipping outliers degrades training stability and performance. They explore mitigation strategies including absorbing outliers into learnable parameters via PreAffine and introducing explicit gating with GatedNorm, which reduces activation magnitudes, preserves or improves performance, and enhances quantization robustness, especially under aggressive FP4 settings. The results demonstrate reduced sensitivity to architectural choices, smoother activations, and practical gains in deployment, providing a unified framework for understanding and mitigating outliers in transformers. This approach has significant implications for training stability and quantization robustness in large-scale models and points to practical, scalable techniques for robust transformer training.

Abstract

We investigate the functional role of emergent outliers in large language models, specifically attention sinks (a few tokens that consistently receive large attention logits) and residual sinks (a few fixed dimensions with persistently large activations across most tokens). We hypothesize that these outliers, in conjunction with the corresponding normalizations (\textit{e.g.}, softmax attention and RMSNorm), effectively rescale other non-outlier components. We term this phenomenon \textit{outlier-driven rescaling} and validate this hypothesis across different model architectures and training token counts. This view unifies the origin and mitigation of both sink types. Our main conclusions and observations include: (1) Outliers function jointly with normalization: removing normalization eliminates the corresponding outliers but degrades training stability and performance; directly clipping outliers while retaining normalization leads to degradation, indicating that outlier-driven rescaling contributes to training stability. (2) Outliers serve more as rescale factors rather than contributors, as the final contributions of attention and residual sinks are significantly smaller than those of non-outliers. (3) Outliers can be absorbed into learnable parameters or mitigated via explicit gated rescaling, leading to improved training performance (average gain of 2 points) and enhanced quantization robustness (1.2 points degradation under W4A4 quantization).

Figures (8)

• Figure 1: In the first row, all models exhibit varying degrees of attention sinks: the first token produces attention logits significantly larger than those of other tokens, dominating the attention scores. In the second row, Qwen3-235B-A22B shows massive activations, with dimensions 1806 and 1423 of the first token exceeding 1000. Beyond these extreme values, all models display consistent residual sinks: certain fixed dimensions yield persistently higher activations across all tokens compared to others.
• Figure 2: Reduced residual sinks. For the same input, the baseline (left) has dimension 304 consistently produces large activations across all tokens. This phenomenon is substantially mitigated in both the PreAffine (middle) and GatedNorm (right) variants.
• Figure 3: SwiGLU and GLU with different rescaling method.
• Figure 4: Loss and outliers of MoE-24B-A3B models.
• Figure 5: RMSNorm weights for baseline.