Video Reasoning without Training

TL;DR

Video reasoning with Large Multimodal Models currently relies on costly training-time methods (CoT/SFT, RL) and produces verbose reasoning traces. This work analyzes the entropy dynamics of model outputs and identifies macro and micro exploration/exploitation cycles as the mechanism behind effective reasoning, then introduces V-Reason: a training-free, inference-time optimization that injects a small Value-Cache Controller to modulate internal token distributions via an entropy-based objective. The approach includes a lite variant (V-Reason(Lite)) that prunes KV-cache tokens to reduce memory and computation. Across six video reasoning benchmarks and multiple model scales, V-Reason narrows the gap to RL-trained models to within 0.6% average accuracy and reduces output tokens by up to 58.6%, while also improving inference efficiency; theoretical analyses guarantee bounded entropy updates and robust oscillation control.$

Abstract

Video reasoning using Large Multimodal Models (LMMs) relies on costly reinforcement learning (RL) and verbose chain-of-thought, resulting in substantial computational overhead during both training and inference. Moreover, the mechanisms that control the thinking process in these reasoning models are very limited. In this paper, using entropy of the model's output as a signal, we discover that the high-quality models go through a series of micro-explorations and micro-exploitations which keep the reasoning process grounded (i.e., avoid excessive randomness while the model is exploring or thinking through an answer). We further observe that once this "thinking" process is over, more accurate models demonstrate a better convergence by reducing the entropy significantly via a final exploitation phase (i.e., a more certain convergence towards a solution trajectory). We then use these novel, theoretically-grounded insights to tune the model's behavior directly at inference, without using any RL or supervised fine-tuning. Specifically, during inference, our proposed approach called V-Reason (Video-Reason) adapts the value cache of the LMM via a few optimization steps on a small, trainable controller using an entropy-based objective, i.e., no supervision from any dataset or RL is necessary. This tuning improves the model's micro-exploration and exploitation behavior during inference. Our experiments show that our proposed method achieves significant improvements over the base instruction-tuned models across several video reasoning datasets, narrowing the gap with RL-trained models to within 0.6% average accuracy without any training, while offering massive efficiency benefits: output tokens are reduced by 58.6% compared to the RL model.

Figures (17)

• Figure 1: V-Reason Overview: (a) Entropy of the output distribution averaged over the MMVU zhao2025mmvu dataset of 625 videos. We see clear macro-exploration and macro-exploitation phases with bigger, more accurate models showing lower overall entropy (lower and later peak, followed by a lower final entropy during the macro-exploitation). We use these key insights to adapt a model's behavior in a training-free way using an inference-time optimization technique. (b) Applying V-Reason on Qwen2.5-VL-7B-Instruct makes its entropy behave more similarly to the larger or the RL-trained Video-R1-7B model. (c) Our method achieves higher accuracy than the base LMM and bridges the accuracy gap with the RL model. (d) V-Reason also significantly reduces the total output tokens compared to all models due to a dedicated entropy minimization phase.
• Figure 2: (a) Proposed approach for enhancing video reasoning in a training-free manner using entropy-based objective. V-Reason uses an inference optimization method to modulate the values cache of the last decoder layer with an entropy switching loss ($\mathcal{L}_{switch}$) to further enhance the video reasoning performance. (b) The average entropy plot for Qwen-2.5-VL-7B on the MMVU dataset along with its EMA. The inset depicts the shift in the entropy maxima for the EMA curve denoted by the black arrow (c) EMA entropy plot of V-Reason for a single sample that shows the micro-exploration and micro-exploitation within the macro-exploration phase before the entropy maxima and macro-exploitation phase after. (d) Plot showing the $\alpha_k$ switching in V-Reason for the corresponding example in (c) that ensures bounded entropy updates without a rapid increase.
• Figure 2: Large model results on MMVU.
• Figure 3: Comparison with Alternative Decoding Methods.
• Figure 3: Qualitative result: An example output and comparison with the baseline Qwen-2.5-VL-7B together with its entropy plot shown on the top right. The black arrow in the entropy plot denotes the shift in the EMA peak demonstrating longer exploration for V-Reason compared to the baseline. See other results in \ref{['sec:qualitative_appendix']}.

Theorems & Definitions (6)

• Proposition 1: Bounded entropy updates
• Proposition 2: EMA smoothing bounds oscillations
• Proposition 1: Bounded entropy updates
• proof
• Proposition 2: EMA smoothing bounds oscillations
• proof : Proof