Training instability in deep learning follows low-dimensional dynamical principles

TL;DR

This work reframes training stability in deep learning as a low-dimensional, dynamical property that can be audited across domains. It introduces a four-axis perturbation taxonomy and StabilityBench to systematically probe training trajectories, revealing a pervasive stability–performance dissociation, cross-domain instability manifolds, and predictive pre-collapse signals captured by a learned meta-state. The study shows that stochasticity acts as a stabilizing resource and that meta-state trajectories enable retrospective auditing and identifiability testing without deploying real-time stabilizers. Collectively, these findings provide a principled framework for stability-aware scaling and a foundation for scientifically studying learning dynamics beyond final performance metrics.

Abstract

Deep learning systems achieve remarkable empirical performance, yet the stability of the training process itself remains poorly understood. Training unfolds as a high-dimensional dynamical system in which small perturbations to optimization, data, parameters, or learning signals can induce abrupt and irreversible collapse, undermining reproducibility and scalability. We propose a unified dynamical perspective that characterizes training stability as an intrinsic property of learning systems, organized along four interacting dimensions: optimization, environmental/data, parametric, and learning-signal stability. We operationalize this perspective through controlled perturbation auditing of training trajectories, probing how learning dynamics respond to structured disturbances without modifying learning algorithms. Across reinforcement learning and large language model training, we identify three recurring regularities: high final performance is frequently decoupled from training stability; controlled stochasticity consistently buffers learning dynamics across paradigms; and deviations in low-dimensional latent meta-states systematically precede observable performance collapse. Together, these findings establish training stability as a measurable and comparable dynamical property of learning systems, providing a descriptive foundation for studying learning dynamics beyond final performance outcomes.

Figures (3)

• Figure 1: A unifying dynamical framework for auditing dynamical reliability beyond capability scaling.(a) Training as a non-autonomous dynamical system. Learning is viewed as a non-autonomous dynamical process in which the policy, data distribution, optimization state, and environment co-evolve over training, giving rise to diverse instability modes that cannot be captured by static performance metrics alone. (b) Meta-state as a structural representation. The meta-state provides a low-dimensional structural representation of the evolving learning dynamics, aggregating multiple signals into a joint state that characterizes proximity to instability, rather than serving as an individual diagnostic metric. (c) Perturbation-based auditing and conditional interaction. Building on this representation, perturbation-based auditing probes dynamical responses under controlled disturbances and enables conditional interaction with training, providing empirical support that the meta-state functions as a structural variable whose deviations can be used for stability-aware monitoring and conditional probing.
• Figure 2: Algorithm-specific catastrophic instability under optimization perturbations. A single, localized learning-rate spike induces abrupt and irreversible training collapse in PPO, despite comparable pre-perturbation performance, whereas SAC and TD3 remain stable. This demonstrates that training instability manifests as an algorithm-dependent failure mode rather than gradual performance degradation or noise accumulation. Additional perturbation analyses are reported in Supplementary Fig. S1.
• Figure 3: Structural and dynamical signatures of training instability across learning paradigms. All trajectories are aligned such that Step = 0 corresponds to perturbation injection. (a) Gradient directional coherence ($x_{\mathrm{grad}}$) exhibits a sharp and highly consistent collapse across perturbation types and model scales, indicating the loss of coordinated update geometry and the crossing of a shared structural instability boundary. (b) Instantaneous instability ($x_{\mathrm{inst}}$) characterizes the subsequent dynamical unfolding following boundary crossing, including transient amplification and relaxation behavior, and displays substantial heterogeneity across perturbations.