Test-time Prompt Intervention

TL;DR

This work addresses the redundancy and unreliability of chain-of-thought reasoning in large language models by introducing Test-time Prompt Intervention (PI). PI provides a plug-in interface with When, How, and Which modules to dynamically steer CoT during inference, incorporating human expertise and cognitive-science principles. Empirical results across multiple models and benchmarks show substantial CoT compression (≈$50\%$ shorter) and reduced hallucinations (≈$2.5$–$4.1\%$) while preserving or improving accuracy. The approach offers a practical, scalable path to more controllable and interpretable reasoning in LLMs, with potential applications in human–AI collaboration and further training-time trajectory design.

Abstract

Test-time compute has led to remarkable success in the large language model (LLM) community, particularly for complex tasks, where longer chains of thought (CoTs) are generated to enhance reasoning capabilities. However, growing evidence reveals that such reasoning models often produce CoTs plagued by excessive redundancy, including unnecessary verification steps and repetitive reasoning shifts. The root cause lies in post-training of them that overly rely on outcome reward paradigms, as the data of process reward paradigms, which regulate intermediate reasoning steps, is difficult to construct at scale. To address this, we propose PI, a novel framework for Test-time Prompt Intervention. PI provides an interface to dynamically guide and regulate reasoning paths during inference through timely (When module) and proper (How module) interventions and post-intervention sampling (Which module). This allows human problem-solving expertise and cognitive science principles to be seamlessly integrated into LLMs' reasoning processes, enhancing controllability and interpretability. Extensive experiments across multiple models and datasets demonstrate that PI significantly shortens CoTs while reducing hallucination, yielding more concise and reliable reasoning.

Figures (20)

• Figure 1: LRMs' original reasoning misses the optimal trajectory due to overthinking, resulting in verbosity, while $\pi$ 's timely interventions streamline reasoning process, reaching the correct conclusion more efficiently (12 steps to 4 steps).
• Figure 2: An illustrative example showing that LRMs exhibit confused and redundant thought processes during reasoning. (a) Step-level attention map visualization of a complete reasoning trajectory, with steps separated by "\\ n\\ n". (b) Directed Acyclic Graph (DAG) representation of the reasoning process, where each step is a node and edge thickness reflects the magnitude of attention values. (c) Abbreviated content of each reasoning step. More details of experimental setups are placed in Appendix A.
• Figure 3: (a) The word cloud visualization of the LRMs' CoTs. (b) The number of verification steps for correctly answered versus incorrectly answered samples. (c) The distribution of the proportion of verification steps in correct answers versus incorrect answers. (d) Accuracy and length of Qwen3-8B-generated response on two datasets under different processing strategies.
• Figure 4: An overview of the Prompt Intervention ($\pi$) framework. See cases in Appendix B for detailed illustration.
• Figure 5: Comparison of experimental results on Qwen3-4B between original generation, static PI, and dynamic PI.