Less is More: Improving LLM Reasoning with Minimal Test-Time Intervention

TL;DR

The paper tackles the inefficiency of test-time reasoning by showing that errors concentrate in a small set of high-entropy tokens. It introduces Minimal Test-Time Intervention (MTI), a training-free framework that applies classifier-free guidance selectively at high-entropy positions and uses lightweight negative-prompt guidance via KV-cache reuse to approximate the unconditional distribution. MTI yields robust improvements across general, coding, and STEM benchmarks (e.g., +$9.28\%$ on six tasks with DeepSeek-R1-7B and +$11.25\%$ on AIME2024 with Ling-mini-2.0), with modest CFG usage, and extends to vision-language models as a plug-and-play enhancement compatible with existing decoding strategies and accelerators.

Abstract

Recent progress in large language models (LLMs) has focused on test-time scaling to improve reasoning via increased inference computation, but often at the cost of efficiency. We revisit test-time behavior and uncover a simple yet underexplored phenomenon: reasoning uncertainty is highly localized-only a small subset of high-entropy tokens dominantly affects output correctness. Motivated by this, we propose Minimal Test-Time Intervention (MTI), a training-free framework that enhances reasoning accuracy and stability with minimal overhead. MTI includes: (i) Selective CFG intervention, applying classifier-free guidance only at uncertain positions; and (ii) Lightweight negative-prompt guidance, reusing the main model's KV cache to approximate unconditional decoding efficiently. MTI yields consistent gains across general, coding, and STEM tasks-e.g., +9.28% average improvement on six benchmarks for DeepSeek-R1-7B and +11.25% on AIME2024 using Ling-mini-2.0-while remaining highly efficient.

Figures (6)

• Figure 1: Incorrect answers exhibit higher average answer entropy than correct ones, primarily due to high-entropy tokens.
• Figure 2: Comparison between CFG and MTI. MTI applies CFG exclusively to high-entropy tokens and reuses the conditional KV cache via negative prompt injection. This approach not only improves computational efficiency and memory overhead by bypassing the unconditional cache but also achieves a superior approximation of the unconditional distribution, thereby effectively suppressing erroneous content generation.
• Figure 3: Ablation study results on the selection of (a) entropy threshold, (b) $\omega$ in classifier-free guidance and (c) negative prompt in MTI.
• Figure 4: Entropy distributions of tokens categorized by vanilla CFG modification success. x-axis: token entropy; y-axis: token count per bin.
• Figure 5: Token word clouds before vs. after MTI.