Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Spectral Flattening of Quantized Embeddings

Junlin Huang, Wenyi Fang, Zhenheng Tang, Yuxin Wang, Xueze Kang, Yang Zheng, Bo Li, Xiaowen Chu

TL;DR

This work addresses the instability of training large language models at ultra-low precision by linking Zipfian language statistics to the heavy-tailed spectral structure of embeddings. It shows that uniform quantization introduces a noise floor that truncates the spectral tail, paradoxically increasing stable rank and causing representational collapse. The authors derive explicit bounds on top singular values and tail perturbations, and validate these predictions with experiments across GPT-2 variants, demonstrating spectral flattening and eventual collapse under 4-bit quantization. The findings highlight spectral fidelity as a necessary condition for stable low-bit optimization and motivate quantization schemes that respect the intrinsic heavy-tailed spectrum for robust training.

Abstract

Training Large Language Models (LLMs) at ultra-low precision is critically impeded by instability rooted in the conflict between discrete quantization constraints and the intrinsic heavy-tailed spectral nature of linguistic data. By formalizing the connection between Zipfian statistics and random matrix theory, we prove that the power-law decay in the singular value spectra of embeddings is a fundamental requisite for semantic encoding. We derive theoretical bounds showing that uniform quantization introduces a noise floor that disproportionately truncates this spectral tail, which induces spectral flattening and a strictly provable increase in the stable rank of representations. Empirical validation across diverse architectures including GPT-2 and TinyLlama corroborates that this geometric degradation precipitates representational collapse. This work not only quantifies the spectral sensitivity of LLMs but also establishes spectral fidelity as a necessary condition for stable low-bit optimization.

On the Spectral Flattening of Quantized Embeddings

TL;DR

This work addresses the instability of training large language models at ultra-low precision by linking Zipfian language statistics to the heavy-tailed spectral structure of embeddings. It shows that uniform quantization introduces a noise floor that truncates the spectral tail, paradoxically increasing stable rank and causing representational collapse. The authors derive explicit bounds on top singular values and tail perturbations, and validate these predictions with experiments across GPT-2 variants, demonstrating spectral flattening and eventual collapse under 4-bit quantization. The findings highlight spectral fidelity as a necessary condition for stable low-bit optimization and motivate quantization schemes that respect the intrinsic heavy-tailed spectrum for robust training.

Abstract

Training Large Language Models (LLMs) at ultra-low precision is critically impeded by instability rooted in the conflict between discrete quantization constraints and the intrinsic heavy-tailed spectral nature of linguistic data. By formalizing the connection between Zipfian statistics and random matrix theory, we prove that the power-law decay in the singular value spectra of embeddings is a fundamental requisite for semantic encoding. We derive theoretical bounds showing that uniform quantization introduces a noise floor that disproportionately truncates this spectral tail, which induces spectral flattening and a strictly provable increase in the stable rank of representations. Empirical validation across diverse architectures including GPT-2 and TinyLlama corroborates that this geometric degradation precipitates representational collapse. This work not only quantifies the spectral sensitivity of LLMs but also establishes spectral fidelity as a necessary condition for stable low-bit optimization.
Paper Structure (35 sections, 16 theorems, 57 equations, 13 figures, 4 tables)

This paper contains 35 sections, 16 theorems, 57 equations, 13 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Contribution 1:
  3. Contribution 2:
  4. Preliminaries and Theoretical Foundations
  5. Asymptotic Notations
  6. Statistical Structure of Language Representations
  7. Tools for Spectral Perturbation Analysis
  8. Low-rank Nature of Language Model Embedding
  9. Population Covariance and Spectral Decomposition
  10. Matrix Phase Transitions
  11. Singular Values Bounds
  12. Quantization Increases Stable Rank
  13. Problem Setup and Spectral Assumptions
  14. Bounding the Spectral Norm of Noise
  15. Relative Error Analysis of Singular Values
  16. ...and 20 more sections

Key Result

Theorem 2.5

Let $\{\mu_n\}_{n \ge 1}$ be a sequence of probability measures on $\mathbb{R}$, and let $\{m_n(z)\}_{n \ge 1}$ denote their corresponding Stieltjes transforms. If there exists a function $m(z)$ defined on the upper complex plane $\mathbb{C}^+$, such that for all $z \in \mathbb{C}^+$ the pointwise l and $m(z)$ is the Stieltjes transform of a probability measure $\mu$, then $\{\mu_n\}$ converges we

Figures (13)

  • Figure 1: Spectral analysis of MLP embedding matrices. (Left) The ratio of quantized to original singular values across indices. (Right) Logarithmic distribution of singular values with BF16 and NVFP4 quantization.
  • Figure 2: Spectral evolution of training. Top: Initial noise inflates Stable Rank (SR). Bottom: Later steps show representational collapse.
  • Figure 3: The histograms display the frequency of quantization errors for MLP linear layers in the GPT-2-124M model quantized with NVFP4 . The red curves represent normal distribution fits. The annotated $\mu$ values indicate the empirical mean of the error for each layer.
  • Figure 4: The scatter plots illustrate the relationship between the relative quantization error $\epsilon_k$ (y-axis) and the inverse of the singular value $1/\sigma_k$ (x-axis) across varying layers of the GPT-2-124M model. The red dashed lines represent the linear regression fit. The coefficient of determination ($R^2$) is annotated for each layer, indicating the goodness of fit.
  • Figure : GPT-2-124M
  • ...and 8 more figures

Theorems & Definitions (21)

  • Definition 2.1: Asymptotic Equivalence
  • Definition 2.2: Asymptotic Upper Bound
  • Theorem 2.5
  • Theorem 2.6: Weyl
  • Theorem 2.7: Bernstein
  • Lemma 3.1
  • Lemma 3.2
  • Lemma 3.3
  • Lemma 3.4
  • Theorem 3.5: BBP Phase Transition
  • ...and 11 more