Anatomy of Capability Emergence: Scale-Invariant Representation Collapse and Top-Down Reorganization in Neural Networks

TL;DR

This work tracks five geometric measures across five model scales, 120+ emergence events in eight algorithmic tasks, and three Pythia language models and delineates prediction limits: geometric measures encode coarse task difficulty but not fine-grained timing (within-class concordance 27%; when task ordering reverses across scales, prediction fails at 26%).

Abstract

Capability emergence during neural network training remains mechanistically opaque. We track five geometric measures across five model scales (405K-85M parameters), 120+ emergence events in eight algorithmic tasks, and three Pythia language models (160M-2.8B). We find: (1) training begins with a universal representation collapse to task-specific floors that are scale-invariant across a 210X parameter range (e.g., modular arithmetic collapses to RANKME $\approx$ 2.0 regardless of model size); (2) collapse propagates top-down through layers (32/32 task X model consistency), contradicting bottom-up feature-building intuition; (3) a geometric hierarchy in which representation geometry leads emergence (75-100% precursor rate for hard tasks), while the local learning coefficient is synchronous (0/24 precursor) and Hessian measures lag. We also delineate prediction limits: geometric measures encode coarse task difficulty but not fine-grained timing (within-class concordance 27%; when task ordering reverses across scales, prediction fails at 26%). On Pythia, global geometric patterns replicate but per-task precursor signals do not -- the precursor relationship requires task-training alignment that naturalistic pre-training does not provide. Our contribution is the geometric anatomy of emergence and its boundary conditions, not a prediction tool.

Figures (8)

• Figure 1: Emergence map. Log-scaled emergence step for 8 tasks across 5 model sizes. Easy tasks (above white line) emerge uniformly early regardless of scale; hard tasks (below) show scale-dependent acceleration. Color encodes $\log_{10}(\text{emergence step})$ using a sequential colormap.
• Figure 2: Task-specific collapse floors. RankMe trajectories for four hard tasks across five model sizes (405K--85M parameters). All models collapse to task-specific minima during initialization (steps 0--200, shaded), then recover at different rates. MOD shows robust scale invariance (floor $\approx 2.0$, CV = 0.08), while SORT and MUL floors increase with model capacity.
• Figure 3: Top-down layer propagation. Per-layer RankMe at the collapse minimum (left) and post-emergence (right) for three model sizes on MOD. At collapse, deeper layers show lower RankMe (top-down gradient). Post-emergence, the final layer stays compressed while middle layers recover, creating a specialization-diversity gradient.
• Figure 4: Geometric hierarchy. (a) Accuracy (left axis, black), RankMe (right axis, blue), and LLC (right axis, red) for MOD on nano (405K params). Both geometric measures are min-max normalized to [0,1] for visual comparison. RankMe collapses before emergence while LLC rises synchronously with accuracy. (b) Precursor rates across five geometric measures: RankMe leads in 86% of cases; LLC shows 0% precursor signal.
• Figure 5: Limits of geometric prediction. (a) Early RankMe (at collapse floor) versus emergence step, colored by difficulty class. Within each class (easy = circles, hard = squares), RankMe does not reliably separate tasks. (b) Within-class concordance rates. Cross-class separation is strong (88%), but within-class prediction is weak: easy tasks score 27% (below the 50% chance line), and the swap test (predicting ordering changes across scales) fails at 26%.