Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Test-time Recursive Thinking: Self-Improvement without External Feedback

Yufan Zhuang, Chandan Singh, Liyuan Liu, Yelong Shen, Dinghuai Zhang, Jingbo Shang, Jianfeng Gao, Weizhu Chen

TL;DR

This work proposes Test-time Recursive Thinking (TRT), an iterative self-improvement framework that conditions generation on rollout-specific strategies, accumulated knowledge, and self-generated verification signals for large language models to self-improve without the need for additional training.

Abstract

Modern Large Language Models (LLMs) have shown rapid improvements in reasoning capabilities, driven largely by reinforcement learning (RL) with verifiable rewards. Here, we ask whether these LLMs can self-improve without the need for additional training. We identify two core challenges for such systems: (i) efficiently generating diverse, high-quality candidate solutions, and (ii) reliably selecting correct answers in the absence of ground-truth supervision. To address these challenges, we propose Test-time Recursive Thinking (TRT), an iterative self-improvement framework that conditions generation on rollout-specific strategies, accumulated knowledge, and self-generated verification signals. Using TRT, open-source models reach 100% accuracy on AIME-25/24, and on LiveCodeBench's most difficult problems, closed-source models improve by 10.4-14.8 percentage points without external feedback.

Test-time Recursive Thinking: Self-Improvement without External Feedback

TL;DR

This work proposes Test-time Recursive Thinking (TRT), an iterative self-improvement framework that conditions generation on rollout-specific strategies, accumulated knowledge, and self-generated verification signals for large language models to self-improve without the need for additional training.

Abstract

Modern Large Language Models (LLMs) have shown rapid improvements in reasoning capabilities, driven largely by reinforcement learning (RL) with verifiable rewards. Here, we ask whether these LLMs can self-improve without the need for additional training. We identify two core challenges for such systems: (i) efficiently generating diverse, high-quality candidate solutions, and (ii) reliably selecting correct answers in the absence of ground-truth supervision. To address these challenges, we propose Test-time Recursive Thinking (TRT), an iterative self-improvement framework that conditions generation on rollout-specific strategies, accumulated knowledge, and self-generated verification signals. Using TRT, open-source models reach 100% accuracy on AIME-25/24, and on LiveCodeBench's most difficult problems, closed-source models improve by 10.4-14.8 percentage points without external feedback.
Paper Structure (51 sections, 2 equations, 14 figures, 1 table, 1 algorithm)

This paper contains 51 sections, 2 equations, 14 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Meta-Reinforcement Learning and Test-Time Adaptation.
  4. Context Engineering and In-Context Self-Improvement.
  5. Parallel Scaling and Solution Aggregation.
  6. Methods: Test-time Recursive Thinking
  7. (1) Generate.
  8. (2) Select.
  9. (3) Reflect.
  10. Core Mechanisms
  11. Knowledge Representation.
  12. Strategy Design.
  13. Domain Specific Designs
  14. Mathematical Reasoning: Mutual Exclusivity.
  15. Code Generation: Execution-Based Self-Verification.
  16. ...and 36 more sections

Figures (14)

  • Figure 1: LiveCodeBench results. TRT accuracy over 8 rounds with test execution on LiveCodeBench v6 hard problems, compared to RSA with 8 rounds. o4-mini improves from 63.5% to 73.9% (+10.4 pp), exceeding RSA's 70.4%; o3 improves from 57.1% to 71.9% (+14.8 pp), exceeding RSA's 69.7%.
  • Figure 2: Test-time Recursive Thinking iterates between three stages: (1) the model generates a set of rollouts conditioned on the current knowledge list and rollout-specific strategies; (2) it then self-ranks these rollouts and selects the best solution for the current round; (3) finally, the model performs a pairwise analysis comparing the remaining solutions against the best one, distills what to avoid into an updated knowledge list, and synthesizes new rollout strategies to explore in the next round.
  • Figure 3: AIME-25 results. Test-time Recursive Thinking achieves 100% accuracy for both gpt-oss-120b and Qwen3-235B. The rolling majority vote (Majority@Prev) shows monotonic improvement, outperforming the Parallel Thinking baseline (Majority@64).
  • Figure 4: Rollout exploration efficiency comparison. At equivalent sample counts, Test-time Recursive Thinking's strategic planning consistently outperforms independent sampling by 2-7 pp in pass@k across both models.
  • Figure 5: Problem-level and strategy analysis (o4-mini with test execution). (a) Problem attribution by round 8 state: Baseline Correct (baseline solved, TRT retained), TRT New Solve (baseline failed, TRT solved), TRT Unstable (baseline solved, TRT lost), Never Solved (both failed). Categories sum to Selected accuracy (73.9%). (b) Strategy transitions: models switch more frequently after failure (82%) than success (74%). (c) Exploration depth: solve rate increases with the number of unique technique clusters explored.
  • ...and 9 more figures