PACED: Distillation at the Frontier of Student Competence

TL;DR

Paced is a framework that concentrates distillation on the zone of proximal development -- the frontier of a student model's competence -- via a principled pass-rate weight derived from the boundary-vanishing structure of distillation gradients.

Abstract

Standard LLM distillation wastes compute on two fronts: problems the student has already mastered (near-zero gradients) and problems far beyond its reach (incoherent gradients that erode existing capabilities). We show that this waste is not merely intuitive but structurally inevitable: the gradient signal-to-noise ratio in distillation provably vanishes at both pass-rate extremes. This theoretical observation leads to Paced, a framework that concentrates distillation on the zone of proximal development -- the frontier of a student model's competence -- via a principled pass-rate weight $w(p) = p^α(1 - p)^β$ derived from the boundary-vanishing structure of distillation gradients. Key results: (1) Theory: We prove that the Beta kernel $w(p) = p^α(1-p)^β$ is a leading-order weight family arising from the SNR structure of distillation, and that it is minimax-robust -- under bounded multiplicative misspecification, worst-case efficiency loss is only $O(δ^2)$. (2)Distillation: On distillation from a larger teacher to a smaller student model with forward KL, Paced achieves significant gain over the base model, while keeping benchmark forgetting at a low level. (3)Self-distillation: On instruction-tuned models with reverse KL, gains are exceeding baselines as well. (4)Two-stage synergy: A forward-KL-then-reverse-KL schedule yields the strongest results in our setting, reaching substantial improvements on standard reasoning benchmarks -- supporting a mode-coverage-then-consolidation interpretation of the distillation process. All configurations require only student rollouts to estimate pass rates, need no architectural changes, and are compatible with any KL direction.

Figures (3)

• Figure 1: Overview of Paced.Left: The pipeline---an expert provides reference solutions, and the student learns via a distillation loss weighted by pass rate. Right: The Beta-kernel weighting $w(p)=p^\alpha(1-p)^\beta$ concentrates training on the zone of proximal development, suppressing trivial and intractable problems.
• Figure 2: Prompt example for student and teacher policies. Both policies share the same model family but differ in conditioning context. The teacher receives the expert solution $y_{\mathcal{E}}$ as additional context, while the student receives only the original problem. This contextual asymmetry enables black-box expert guidance to be transferred into white-box teacher logits for distillation.
• Figure 3: Empirical gradient SNR vs. student pass rate (Qwen3-8B, forward KL, $K{=}10$ rollouts). Gradients are computed at lm_head. Per-problem SNR values are averaged within each pass-rate bin; bin means are then normalized to $[0,1]$ by dividing by the maximum bin mean. Red bars mark boundary regions ($p<0.2$ or $p>0.8$) where SNR is substantially lower; green bars mark the zone of proximal development where training signal is richest.

Theorems & Definitions (28)

• Definition 1: Learning Signal Quality
• Proposition 1: Non-Monotonicity of Learning Signal
• proof
• Proposition 2: Gradient Boundary Conditions for Distillation
• proof
• Proposition 3: Log-Linear Representation of Boundary-Vanishing Functions
• proof
• Definition 2: Per-Step Guaranteed Descent Rate (Lower Bound on Descent)
• Theorem 4: Per-Problem Descent Maximization Yields Beta Kernel Weights
• proof : Proof of Theorem \ref{['thm:beta_optimal']}