Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TTSR: Test-Time Self-Reflection for Continual Reasoning Improvement

Haoyang He, Zihua Rong, Liangjie Zhao, Yunjia Zhao, Lan Yang, Honggang Zhang

TL;DR

Results show that TTSR consistently improves reasoning performance and generalizes well across different model backbones and general-domain reasoning tasks, and suggest that teacher-mediated self-reflection provides an effective pathway for stable and continual reasoning improvement at test time.

Abstract

Test-time Training enables model adaptation using only test questions and offers a promising paradigm for improving the reasoning ability of large language models (LLMs). However, it faces two major challenges: test questions are often highly difficult, making self-generated pseudo-labels unreliable, and existing methods lack effective mechanisms to adapt to a model's specific reasoning weaknesses, leading to inefficient learning. To address these issues, we propose \textbf{TTSR}, a self-reflective test-time self-evolving training framework. TTSR employs a single pretrained language model that alternates between the roles of a \textit{Student} and a \textit{Teacher} at test time. The Student focuses on solving problems and learning from synthesized variant questions, while the Teacher analyzes the Student's failed reasoning trajectories, summarizes recurring reasoning weaknesses, and synthesizes targeted variant questions accordingly. This process guides the model to improve within a learnable regime through a continual self-evolving loop. Experimental results on multiple challenging mathematical reasoning benchmarks show that TTSR consistently improves reasoning performance and generalizes well across different model backbones and general-domain reasoning tasks. These findings suggest that teacher-mediated self-reflection provides an effective pathway for stable and continual reasoning improvement at test time.

TTSR: Test-Time Self-Reflection for Continual Reasoning Improvement

TL;DR

Results show that TTSR consistently improves reasoning performance and generalizes well across different model backbones and general-domain reasoning tasks, and suggest that teacher-mediated self-reflection provides an effective pathway for stable and continual reasoning improvement at test time.

Abstract

Test-time Training enables model adaptation using only test questions and offers a promising paradigm for improving the reasoning ability of large language models (LLMs). However, it faces two major challenges: test questions are often highly difficult, making self-generated pseudo-labels unreliable, and existing methods lack effective mechanisms to adapt to a model's specific reasoning weaknesses, leading to inefficient learning. To address these issues, we propose \textbf{TTSR}, a self-reflective test-time self-evolving training framework. TTSR employs a single pretrained language model that alternates between the roles of a \textit{Student} and a \textit{Teacher} at test time. The Student focuses on solving problems and learning from synthesized variant questions, while the Teacher analyzes the Student's failed reasoning trajectories, summarizes recurring reasoning weaknesses, and synthesizes targeted variant questions accordingly. This process guides the model to improve within a learnable regime through a continual self-evolving loop. Experimental results on multiple challenging mathematical reasoning benchmarks show that TTSR consistently improves reasoning performance and generalizes well across different model backbones and general-domain reasoning tasks. These findings suggest that teacher-mediated self-reflection provides an effective pathway for stable and continual reasoning improvement at test time.
Paper Structure (43 sections, 17 equations, 2 figures, 6 tables)

This paper contains 43 sections, 17 equations, 2 figures, 6 tables.

Table of Contents

  1. Related Work
  2. Self-Play for LLMs.
  3. Preliminary
  4. Group Relative Policy Optimization (GRPO).
  5. Test-Time Training (TTT).
  6. TTSR: Test-Time Self-Reflection
  7. Student Role.
  8. Teacher Role.
  9. Student: Solving Problems and Test-Time Adaptation
  10. Test-Time Training Set Construction.
  11. Majority Voting Reward.
  12. Teacher: Reflection-Guided Curriculum Synthesis
  13. Reflection on Reasoning Steps.
  14. Reflection-Guided Question Synthesis.
  15. Difficulty Reward.
  16. ...and 28 more sections

Figures (2)

  • Figure 1: Comparison between standard Test-time training and TTSR. Standard test-time training produces noisy reward signals when test questions are too hard for the model. In contrast, TTSR reflects on failed reasoning to generate moderately difficult variants matched to the model’s capability, yielding clearer rewards and more stable test-time updates.
  • Figure 2: Cross-dataset generalization. Generalization Evaluation by training the model at test time on different source datasets.