TTCS: Test-Time Curriculum Synthesis for Self-Evolving

TL;DR

TTCS tackles the two main bottlenecks of test-time training for complex reasoning—unreliable pseudo-labels and lack of learnable samples—by introducing a co-evolving Synthesizer and Solver that are optimized online via Group Relative Policy Optimization. The Synthesizer constructs a capability-aware curriculum of tractable variants for each test question, guided by the Solver’s current frontier, while the Solver learns from self-consistency rewards on a mixture of test and synthetic data. Across rigorous math benchmarks (AMC23, AIME24/25, MATH-500, Minerva, OlympiadBench) and general-domain tasks (MMLU-Pro, SuperGPQA, BBEH), TTCS yields substantial performance gains over Self-Consistency, TTRL, and R-Zero, with pronounced advantages on difficult problems and strong cross-domain generalization. Overall, TTCS demonstrates a scalable, adaptive pathway to autonomous self-evolution in reasoning abilities for large language models.

Abstract

Test-Time Training offers a promising way to improve the reasoning ability of large language models (LLMs) by adapting the model using only the test questions. However, existing methods struggle with difficult reasoning problems for two reasons: raw test questions are often too difficult to yield high-quality pseudo-labels, and the limited size of test sets makes continuous online updates prone to instability. To address these limitations, we propose TTCS, a co-evolving test-time training framework. Specifically, TTCS initializes two policies from the same pretrained model: a question synthesizer and a reasoning solver. These policies evolve through iterative optimization: the synthesizer generates progressively challenging question variants conditioned on the test questions, creating a structured curriculum tailored to the solver's current capability, while the solver updates itself using self-consistency rewards computed from multiple sampled responses on both original test and synthetic questions. Crucially, the solver's feedback guides the synthesizer to generate questions aligned with the model's current capability, and the generated question variants in turn stabilize the solver's test-time training. Experiments show that TTCS consistently strengthens the reasoning ability on challenging mathematical benchmarks and transfers to general-domain tasks across different LLM backbones, highlighting a scalable path towards dynamically constructing test-time curricula for self-evolving. Our code and implementation details are available at https://github.com/XMUDeepLIT/TTCS.

Figures (7)

• Figure 1: Comparison of TTRL and our TTCS: (a) When applied to difficult test questions such as AIME24, TTRL suffers from noisy rewards caused by incorrect majority voting consensus. (b) TTCS synthesizes tractable variants to ensure valid pseudo-labels, providing reliable supervision for stable self-evolution.
• Figure 2: Overview of TTCS, a co-evolving test-time training framework. (a) Synthesizer training: the synthesizer first rollout synthetic questions conditioned on test questions with the prompt template, and then is optimized via GRPO with the question quality reward. (b) Solver training: the solver performs online self-evolving on a mixture of test and synthetic questions with self-supervised rewards, updated via GRPO using the self-consistency reward.
• Figure 3: Generalization analysis. (a) General-domain generalization: Accuracy trends of TTCS and TTRL on general-domain reasoning benchmarks (MMLU-Pro, SuperGPQA) during test-time training on AIME25. The green dashed line indicates the R-Zero baseline. (b) Mathematical-domain generalization: Performance comparison on out-of-distribution (OOD) mathematical benchmarks.
• Figure 4: Data efficiency analysis of test-time training on AIME24.
• Figure 5: The general-domain performance comparison of TTCS and the other baselines when TTCS and TTRL trained on AIME24 dataset.