Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ROM: Real-time Overthinking Mitigation via Streaming Detection and Intervention

Xinyan Wang, Xiaogeng Liu, Chaowei Xiao

Abstract

Large Reasoning Models (LRMs) achieve strong accuracy on challenging tasks by generating long Chain-of-Thought traces, but suffer from overthinking. Even after reaching the correct answer, they continue generating redundant reasoning steps. This behavior increases latency and compute cost and can also lead to answer drift. Existing mitigation methods either require training-heavy backbone modification or rely on hand-crafted heuristics that do not truly capture overthinking patterns. We propose ROM, the first method that formulates overthinking mitigation as a streaming prediction-and-control problem. ROM attaches a lightweight detection head to the late-layer hidden states of a frozen large language model backbone. It monitors tokens in real time and triggers an early transition to the final answer once overthinking is detected. We also introduce token-level supervision based on solution correctness boundaries and a data augmentation strategy that reduces distilled-data bias. Across seven benchmarks, ROM achieves the highest accuracy (93.51%), the shortest responses (1,159 tokens), and the best response efficiency. Compared with the vanilla baseline, it reduces response length by 47.2% and improves efficiency by 121%. These results show that streaming detection is a promising approach to real-time overthinking mitigation.

ROM: Real-time Overthinking Mitigation via Streaming Detection and Intervention

Abstract

Large Reasoning Models (LRMs) achieve strong accuracy on challenging tasks by generating long Chain-of-Thought traces, but suffer from overthinking. Even after reaching the correct answer, they continue generating redundant reasoning steps. This behavior increases latency and compute cost and can also lead to answer drift. Existing mitigation methods either require training-heavy backbone modification or rely on hand-crafted heuristics that do not truly capture overthinking patterns. We propose ROM, the first method that formulates overthinking mitigation as a streaming prediction-and-control problem. ROM attaches a lightweight detection head to the late-layer hidden states of a frozen large language model backbone. It monitors tokens in real time and triggers an early transition to the final answer once overthinking is detected. We also introduce token-level supervision based on solution correctness boundaries and a data augmentation strategy that reduces distilled-data bias. Across seven benchmarks, ROM achieves the highest accuracy (93.51%), the shortest responses (1,159 tokens), and the best response efficiency. Compared with the vanilla baseline, it reduces response length by 47.2% and improves efficiency by 121%. These results show that streaming detection is a promising approach to real-time overthinking mitigation.
Paper Structure (37 sections, 7 equations, 2 figures, 6 tables, 1 algorithm)

This paper contains 37 sections, 7 equations, 2 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. LRMs and Overthinking
  4. Mitigating Overthinking
  5. Streaming Token-Level Detection
  6. Method
  7. Problem Setup
  8. Correctness-Boundary Labeling and Augmentation
  9. Token-Level Labeling via First Correct Solution
  10. Counterfactual Self-Correction: Mitigating First-Solution Bias
  11. Streaming Detection
  12. Feature extraction.
  13. Temporal modeling.
  14. Classification head and training objective.
  15. Boundary-Aware Intervention
  16. ...and 22 more sections

Figures (2)

  • Figure 1: Overview of the ROM framework. ROM attaches a lightweight detection head to a frozen LLM backbone and runs in lockstep with decoding. It consists of three components: (1) correctness-boundary labeling and counterfactual self-correction augmentation, (2) streaming overthinking detection via a compact neural probe, and (3) boundary-aware intervention with early stopping and backtracing.
  • Figure 2: Case study on a GSM8K problem. The vanilla model generates 637 tokens of redundant reasoning after reaching the correct answer. ROM detects overthinking and intervenes, but cuts mid-sentence, causing the model to produce a lengthy explanation after the answer box (117 tokens). ROM$_{\text{CSC}}$ with backtracing rewinds to a clean boundary, yielding a concise 202-token response (75.8% reduction).