Level Up: Defining and Exploiting Transitional Problems for Curriculum Learning

Abstract

Curriculum learning--ordering training examples in a sequence to aid machine learning--takes inspiration from human learning, but has not gained widespread acceptance. Static strategies for scoring item difficulty rely on indirect proxy scores of varying quality and produce curricula that are not specific to the learner at hand. Dynamic approaches base difficulty estimates on gradient information, requiring considerable extra computation during training. We introduce a novel method for measuring the difficulty of individual problem instances directly relative to the ability of a given model, and identify transitional problems that are consistently easier as model ability increases. Applying this method to chess and mathematics, we find that training on a curriculum that "levels up" from easier to harder transitional problems most efficiently improves a model to the next tier of competence. These problems induce a natural progression from easier to harder items, which outperforms other training strategies. By measuring difficulty directly relative to model competence, our method yields interpretable problems, learner-specific curricula, and a principled basis for step-by-step improvement.

Figures (22)

• Figure 1: Transitional Problems at a level $i$ can only and consistently be solved by models at a competence level $j\ge i$. We find that training on the next-level transitional problems most efficiently “levels up” a model to the next competence level, which induces a natural ascending curriculum that can be compared to other training strategies.
• Figure 2: Curriculum learning results on Math (left) and Chess (right). We compare curricula based on our transitional point difficulty measure (Level-Down, IID, Level-Up) against curricula using external difficulty measures. IID* denotes random ordering over the training set, serving as the baseline for external measures. For math, external measures include problem length and number of reasoning steps; for chess, number of legal moves, solution length (principal variation depth), and human rating from Lichess. Level-Up with transitional points outperforms all baselines in both domains.
• Figure 3: Math: Cross-model transfer results, for a Qwen2.5-0.5B trained on transitional problems from the Qwen2.5 model family. The level-up curriculum (corresponding to progressive distillation) performs the best.
• Figure 4: Math: Cross dataset transfer results, from the GSM8k to Orca datasets. Qwen2.5-0.5B trained on level-up and level-down curricula and random neo-transitional problems from the Orca dataset are compared to baseline curricula on the Orca training set.
• Figure 5: Chess: Transfer from positions to puzzles. Models trained on $\mathcal{D}^{pos}$ with transitional point curricula, evaluated on $\mathcal{D}^{puz}$. Level-Up with transitional points outperforms all baselines.

Theorems & Definitions (3)

• Definition 2.1: Model Series
• Definition 2.2: Transitional Problem
• Definition 2.3: Transitional Problems at $\tau$