Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Excess Description Length of Learning Generalizable Predictors

Elizabeth Donoway, Hailey Joren, Fabien Roger, Jan Leike

TL;DR

The paper develops Excess Description Length (EDL), an information-theoretic measure defined via prequential MDL, to quantify how much predictive structure fine-tuning absorbs from training data. It formalizes EDL, proves non-negativity in expectation and convergence to surplus description length in the infinite-data limit, and derives a bound linking EDL to generalization improvements. Through toy models, it clarifies when EDL signatures indicate latent elicitation versus teaching, including phenomena like diagnostic single-example gains and coupon-collector-like phase transitions. The framework provides practical, computable diagnostics for empirical studies of fine-tuning, with explicit normalizations and a communication-protocol interpretation, while acknowledging limitations such as algorithm dependence and non-semantic interpretation of the absorbed information.

Abstract

Understanding whether fine-tuning elicits latent capabilities or teaches new ones is a fundamental question for language model evaluation and safety. We develop a formal information-theoretic framework for quantifying how much predictive structure fine-tuning extracts from the train dataset and writes into a model's parameters. Our central quantity, Excess Description Length (EDL), is defined via prequential coding and measures the gap between the bits required to encode training labels sequentially using an evolving model (trained online) and the residual encoding cost under the final trained model. We establish that EDL is non-negative in expectation, converges to surplus description length in the infinite-data limit, and provides bounds on expected generalization gain. Through a series of toy models, we clarify common confusions about information in learning: why random labels yield EDL near zero, how a single example can eliminate many bits of uncertainty about the underlying rule(s) that describe the data distribution, why structure learned on rare inputs contributes proportionally little to expected generalization, and how format learning creates early transients distinct from capability acquisition. This framework provides rigorous foundations for the empirical observation that capability elicitation and teaching exhibit qualitatively distinct scaling signatures.

Excess Description Length of Learning Generalizable Predictors

TL;DR

The paper develops Excess Description Length (EDL), an information-theoretic measure defined via prequential MDL, to quantify how much predictive structure fine-tuning absorbs from training data. It formalizes EDL, proves non-negativity in expectation and convergence to surplus description length in the infinite-data limit, and derives a bound linking EDL to generalization improvements. Through toy models, it clarifies when EDL signatures indicate latent elicitation versus teaching, including phenomena like diagnostic single-example gains and coupon-collector-like phase transitions. The framework provides practical, computable diagnostics for empirical studies of fine-tuning, with explicit normalizations and a communication-protocol interpretation, while acknowledging limitations such as algorithm dependence and non-semantic interpretation of the absorbed information.

Abstract

Understanding whether fine-tuning elicits latent capabilities or teaches new ones is a fundamental question for language model evaluation and safety. We develop a formal information-theoretic framework for quantifying how much predictive structure fine-tuning extracts from the train dataset and writes into a model's parameters. Our central quantity, Excess Description Length (EDL), is defined via prequential coding and measures the gap between the bits required to encode training labels sequentially using an evolving model (trained online) and the residual encoding cost under the final trained model. We establish that EDL is non-negative in expectation, converges to surplus description length in the infinite-data limit, and provides bounds on expected generalization gain. Through a series of toy models, we clarify common confusions about information in learning: why random labels yield EDL near zero, how a single example can eliminate many bits of uncertainty about the underlying rule(s) that describe the data distribution, why structure learned on rare inputs contributes proportionally little to expected generalization, and how format learning creates early transients distinct from capability acquisition. This framework provides rigorous foundations for the empirical observation that capability elicitation and teaching exhibit qualitatively distinct scaling signatures.
Paper Structure (61 sections, 28 theorems, 111 equations, 8 figures)

This paper contains 61 sections, 28 theorems, 111 equations, 8 figures.

Table of Contents

  1. Introduction
  2. Information-Theoretic Background and Definitions
  3. Setup and Notation
  4. Cross-Entropy Loss as Compression
  5. Prequential Coding
  6. Minimum Description Length
  7. Surplus Description Length
  8. Formal Definitions
  9. Prequential Minimum Description Length
  10. Excess Description Length
  11. Interpretation as a Communication Protocol
  12. Normalizations
  13. Relationship to Surplus Description Length
  14. Properties and Bounds
  15. Non-Negativity
  16. ...and 46 more sections

Key Result

Theorem 4.1

Let $D$ be drawn i.i.d. from distribution $\mathcal{D}$, let the test loss be evaluated on an independent sample $D_\text{test}$ from $\mathcal{D}$, and let $A$ be a population-monotonic algorithm (def:populationmonotonic). Then

Figures (8)

  • Figure 1: When labels are random, test loss $L_\text{test}$ remains constant since no generalizable pattern can be learned. MDL and the residual codelength $n \cdot L_\text{test}$ both increase linearly (in expectation), proportional to the number of training examples $n$. Since no learnable structure exists in the data, $\text{EDL} = \text{MDL} - n \cdot L_\text{test}\approx 0$, and the amount of generalizable information absorbed is negligible.
  • Figure 2: In the low-coverage regime (Phase 1, green), additional examples provide opportunities to learn generalizable patterns from relatively few known concepts. The rate of information absorption peaks as full coverage of concepts is approached (Phase 2, yellow). Beyond the coverage threshold (peak), learning enters the full-coverage regime (Phase 3, red), as additional examples yield diminishing predictive information that can be absorbed.
  • Figure 3: Format learning can induce transients in EDL scaling and dynamical behavior. In teaching scenarios, format learning can cause a transient drop in the per-example EDL ($\text{EDL}/n$), as format comprises generalizable structure that can be easily learned before task capability is acquired.
  • Figure 4: When labels are random, test loss $L_\text{test}$ remains constant since no generalizable pattern can be learned. MDL and the residual codelength $n \cdot L_\text{test}$ both increase linearly (in expectation), proportional to the number of training examples $n$. Since no learnable structure exists in the data, $\text{EDL} = \text{MDL} - n \cdot L_\text{test}\approx 0$, and the amount of generalizable information absorbed is negligible.
  • Figure 5: A single example can contribute up to $\log m$ bits of generalizable information by reducing uncertainty about which hypothesis is correct.
  • ...and 3 more figures

Theorems & Definitions (50)

  • Definition 1: Prequential MDL
  • Definition 2: Excess Description Length
  • Definition 3: Population-Monotonic Algorithm
  • Theorem 4.1: Non-negativity in Expectation
  • Remark 1
  • proof
  • Theorem 4.2: MDL and Regret
  • Corollary 1
  • Theorem 4.3: Convergence to SDL
  • Remark 2
  • ...and 40 more