Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stability as a Liability:Systematic Breakdown of Linguistic Structure in LLMs

Xianzhe Meng, Qiangsheng Zeng, Ling Luo, Qinghan Yang, Jiarui Hao, Wenbo Wu, Qinyu Wang, Rui Yin, Lin Qi, Renzhi Lu

TL;DR

This work empirically validate how stabilizing training dynamics affects the induced generation distribution and indicates that optimization stability and generative expressivity are not inherently aligned, and that stability alone is an insufficient indicator of generative quality.

Abstract

Training stability is typically regarded as a prerequisite for reliable optimization in large language models. In this work, we analyze how stabilizing training dynamics affects the induced generation distribution. We show that under standard maximum likelihood training, stable parameter trajectories lead stationary solutions to approximately minimize the forward KL divergence to the empirical distribution, while implicitly reducing generative entropy. As a consequence, the learned model can concentrate probability mass on a limited subset of empirical modes, exhibiting systematic degeneration despite smooth loss convergence. We empirically validate this effect using a controlled feedback-based training framework that stabilizes internal generation statistics, observing consistent low-entropy outputs and repetitive behavior across architectures and random seeds. It indicates that optimization stability and generative expressivity are not inherently aligned, and that stability alone is an insufficient indicator of generative quality.

Stability as a Liability:Systematic Breakdown of Linguistic Structure in LLMs

TL;DR

This work empirically validate how stabilizing training dynamics affects the induced generation distribution and indicates that optimization stability and generative expressivity are not inherently aligned, and that stability alone is an insufficient indicator of generative quality.

Abstract

Training stability is typically regarded as a prerequisite for reliable optimization in large language models. In this work, we analyze how stabilizing training dynamics affects the induced generation distribution. We show that under standard maximum likelihood training, stable parameter trajectories lead stationary solutions to approximately minimize the forward KL divergence to the empirical distribution, while implicitly reducing generative entropy. As a consequence, the learned model can concentrate probability mass on a limited subset of empirical modes, exhibiting systematic degeneration despite smooth loss convergence. We empirically validate this effect using a controlled feedback-based training framework that stabilizes internal generation statistics, observing consistent low-entropy outputs and repetitive behavior across architectures and random seeds. It indicates that optimization stability and generative expressivity are not inherently aligned, and that stability alone is an insufficient indicator of generative quality.
Paper Structure (30 sections, 22 theorems, 185 equations, 5 figures, 1 table, 5 algorithms)

This paper contains 30 sections, 22 theorems, 185 equations, 5 figures, 1 table, 5 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Problem Setting
  4. Stationarity and KL Minimization
  5. Stability-Induced Stationarity
  6. Forward KL Minimization
  7. Impact of Approximate KL Optimality
  8. Mode Collapse Induced by Training Stability
  9. Experiments
  10. Experimental Setup
  11. Main Experiments
  12. Ablation Studies
  13. Loss-Entropy Trajectory and Mode Collapse Vulnerability
  14. Conclusion
  15. Definitions and Notations
  16. ...and 15 more sections

Key Result

Lemma 4.1

Every accumulation point $\theta^\ast$ of the SGD trajectory satisfies

Figures (5)

  • Figure 1: Framework for studying mode collapse: feedback network with BARRA module.
  • Figure 2: Ablation Studies
  • Figure 3: Relationship between Loss and Entropy in LLM Training
  • Figure 4: This is a comparison between DMU and DTU, showing its inner consistency in allocating sparsity.
  • Figure :

Theorems & Definitions (35)

  • Definition 3.1: Training Stability
  • Definition 3.2: Mode Collapse
  • Lemma 4.1: Stationarity of Accumulation Points
  • Lemma 4.2: MLE Loss and KL Equivalence
  • Lemma 4.3: KL-Minimizing Properties of Stationary Points
  • Theorem 5.5: Mode collapse under stable training
  • Definition 1.1: Learning Accuracy $\widetilde{a}$
  • Definition 1.2: Learning Complexity $\widetilde{c}$
  • Definition 1.3: Edge-level Dropout
  • Definition 1.4: Node-level Dropout
  • ...and 25 more