Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Preventing Curriculum Collapse in Self-Evolving Reasoning Systems

Vaibhav Mishra

Abstract

Self-evolving reasoning frameworks let LLMs improve their reasoning capabilities by iteratively generating and solving problems without external supervision, using verifiable rewards. Ideally, such systems are expected to explore a diverse problem space and propose new challenges of high learning value. While prior work has largely focused on solver-side optimisation and verification, recent evidence suggests that self-evolving systems can exhibit diversity collapse in posing new problems after just a few iterations, even when surface-level variation is preserved. We introduce Prism, a question-centric self-evolution method that directly tackles this collapse. Prism defines a persistent diversity signal over an embedding-induced semantic partition of mathematical problems and uses it to encourage balanced exploration of underrepresented regions across iterations. This coverage signal is combined with a Zone-of-Proximal-Development (ZPD) gate to preserve edge-of-solvability difficulty. Evaluated on seven widely used mathematical reasoning benchmarks against five self-evolving baselines, Prism achieves the highest accuracy on six out of seven tasks, achieving gains of +3.98 absolute points over R-Zero on AMC and +3.68 on Minerva Math. Prism also generates semantically diverse and challenging questions across iterations, resulting in the construction of the Prism-Math dataset comprising 100k mathematical questions. These results demonstrate that cross-iteration semantic coverage is a high-leverage and under-explored axis for building more capable self-evolving reasoners. We release the code, dataset, and models to facilitate further research.

Preventing Curriculum Collapse in Self-Evolving Reasoning Systems

Abstract

Self-evolving reasoning frameworks let LLMs improve their reasoning capabilities by iteratively generating and solving problems without external supervision, using verifiable rewards. Ideally, such systems are expected to explore a diverse problem space and propose new challenges of high learning value. While prior work has largely focused on solver-side optimisation and verification, recent evidence suggests that self-evolving systems can exhibit diversity collapse in posing new problems after just a few iterations, even when surface-level variation is preserved. We introduce Prism, a question-centric self-evolution method that directly tackles this collapse. Prism defines a persistent diversity signal over an embedding-induced semantic partition of mathematical problems and uses it to encourage balanced exploration of underrepresented regions across iterations. This coverage signal is combined with a Zone-of-Proximal-Development (ZPD) gate to preserve edge-of-solvability difficulty. Evaluated on seven widely used mathematical reasoning benchmarks against five self-evolving baselines, Prism achieves the highest accuracy on six out of seven tasks, achieving gains of +3.98 absolute points over R-Zero on AMC and +3.68 on Minerva Math. Prism also generates semantically diverse and challenging questions across iterations, resulting in the construction of the Prism-Math dataset comprising 100k mathematical questions. These results demonstrate that cross-iteration semantic coverage is a high-leverage and under-explored axis for building more capable self-evolving reasoners. We release the code, dataset, and models to facilitate further research.
Paper Structure (42 sections, 15 equations, 3 figures, 6 tables, 1 algorithm)

This paper contains 42 sections, 15 equations, 3 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Reinforcement learning for reasoning.
  4. Self-evolving curricula and diversity collapse.
  5. Curriculum learning, problem synthesis, and quality-diversity.
  6. Preliminaries
  7. Notation.
  8. Solvability scoring, policy optimisation, and ZPD gating.
  9. Self-evolving questioner--solver loop.
  10. Semantic partition and EMA coverage tracking.
  11. Coverage-aware questioner objective.
  12. Method
  13. Offline Cluster Space Construction
  14. Cluster-Diversity Reward
  15. Step 1: Cluster assignment.
  16. ...and 27 more sections

Figures (3)

  • Figure 1: Prism performance overview. (a) Pass@1 accuracy comparison with baseline across benchmarks. (b) Math-500 test accuracy across iterations.
  • Figure 2: Iteration-wise progression of the Prism self-evolution loop.
  • Figure 3: Curriculum diversity visualisation. Left: Per-cluster frequency distributions. R-Zero (top) is dominated by a single spike at cluster 44 (951 questions); Prism (bottom) covers 107 of 128 clusters with no dominant mode. Right: Lorenz curves for R-Zero (blue) and Prism (orange). A curve closer to the diagonal indicates more equitable cluster coverage.