Video-R2: Reinforcing Consistent and Grounded Reasoning in Multimodal Language Models

TL;DR

Video-R2 addresses the gap where multimodal video reasoning models produce plausible but poorly grounded traces. It introduces two diagnostics, Think-Answer Consistency (TAC) and Video Attention Score (VAS), and harnesses a Temporal Alignment Reward (TAR) with a consistency gate to train a timestamp-aware model via SFT and GRPO. TAR aligns predicted reasoning timestamps with reference events, gated by reasoning-answer consistency, improving temporal precision and grounding. Across 11 benchmarks, Video-R2 achieves higher TAC, VAS, and accuracy than prior methods, demonstrating that explicit temporal alignment and coherent reasoning yield more trustworthy video understanding. A timestamp-aligned dataset and targeted ablations underscore the value of data quality and the proposed training objectives.

Abstract

Reasoning over dynamic visual content remains a central challenge for multimodal large language models. Recent thinking models generate explicit reasoning traces for interpretability; however, their reasoning often appears convincing while being logically inconsistent or weakly grounded in visual evidence. We identify and formalize these issues through two diagnostic metrics: Think Answer Consistency (TAC), which measures the alignment between reasoning and answers, and Video Attention Score (VAS), which captures the extent to which reasoning depends on visual versus textual cues. Analysis across 11 video reasoning benchmarks shows that current models rely heavily on linguistic priors rather than visual content. To address this, we propose a reinforcement learning approach that enhances both temporal precision and reasoning consistency. Our approach combines timestamp aware supervised fine tuning with Group Relative Policy Optimization (GRPO) guided by a novel Temporal Alignment Reward (TAR). This dual step post training stage encourages temporally aligned and causally coherent video reasoning. The resulting model, Video R2, achieves consistently higher TAC, VAS, and accuracy across multiple benchmarks, demonstrating that improvements in temporal alignment and reasoning coherence lead to more accurate and trustworthy video understanding. Code: https://github.com/mbzuai-oryx/Video-R2

Figures (24)

• Figure 1: Inconsistent reasoning in prior video-LLMs and improved visual reliance with Video-R2. Given the video and the question, "How many transactions does Visa have in one day?", both Video-R1videor12025 and VideoChat-R1videochatR12025 conclude option A during their reasoning but finally predict option E as answer, revealing inconsistent reasoning behavior where the model’s conclusion and final answer mismatch. This behaviour emerges because these models primarily rely on textual context and world knowledge, attending little to the video itself. In contrast, our Video-R2 correctly identifies the on-screen visual cue at 01:45 ("23,666 transactions/sec"), performs temporal conversion, and arrives at the correct daily value. The box plot on the right shows the average attention from generated tokens to video tokens across all attention heads in the final transformer layer. Compared with baselines, Video-R2 allocates higher and more distributed attention to video tokens, indicating stronger and more adaptive visual reliance. While prior models can produce plausible yet inconsistent reasoning, Video-R2 reasons coherently and depends on actual video evidence. (Additional examples in Appendix\ref{['supp:tac']}, \ref{['supp:vas']}).
• Figure 2: Comparison of Video-R2 with recent video reasoning models, Video-R1 videor12025, VideoChat-R1/1.5 videochatR12025yan2025videochat_r1.5, and VideoRFT videorft2025, across three metrics: TAC (Think--Answer Consistency), VAS (Video Attention Score), and Accuracy. The upper row reports average scores over six reasoning benchmarks, VideoMathQA rasheed2025videomathqa, Video-MMMU hu2025video_mmmu, MMVU mmvu2024, VSIBench vsibench, MINERVA minerva2024, and SciVideoBench deng2025scivideobench, while the lower row shows averages over all 11 benchmarks including the five generic ones, MVBench li2023mvbench, VideoMME li2023videomme, TempCompass tempcompass, MLVU zhou2025mlvu, and LongVideoBench wu2024longvideobench. Video-R2 performs better across both reasoning and overall evaluations, achieving higher consistency (TAC) and video-focused reasoning (VAS) while maintaining competitive accuracy (Details in Sec. \ref{['sec:experiments']}).
• Figure 3: Temporal Alignment Reward (TAR). The figure illustrates how the proposed TAR is computed for a video reasoning example. We have a reference reasoning containing grounded timestamps and a predicted reasoning generated by our model. Both are processed by an LLM that extracts timestamp-sentence pairs, referred to as claims. The Temporal Match matrix ($T$) checks if a predicted timestamp lies within a temporal tolerance $\Delta$ of a reference timestamp (True if $|t_i^{\mathrm{pred}} - t_j^{\mathrm{ref}}|\le \Delta$, otherwise False). The Semantic Match matrix ($S$) compares sentence similarities using embeddings; a True/tick indicates that the cosine similarity exceeds a threshold $\tau$. A one-to-one bipartite matching is then applied over pairs satisfying both conditions to obtain the binary alignment matrix $X$, ensuring that each predicted claim aligns with at most one reference claim. The precision-based temporal alignment reward is computed as $\mathrm{TAR}_{\text{prec}} = \tfrac{1}{n}\sum_{i,j} X_{ij}$. This score represents the fraction of predicted timestamps that are temporally and semantically valid. Finally, TAR is gated by the consistency indicator $g$ that verifies if the reasoning and answer are coherent, yielding $\mathrm{TAR} = g \times \mathrm{TAR}_{\text{prec}}$. Rows correspond to predictions and columns to references. The example demonstrates how correct temporal matches contribute to higher rewards under logical consistency.
• Figure 4: Reasoning quality comparison on TAC and VAS metrics. The left and right heatmaps show TAC and VAS across 11 benchmarks, including five generic li2023mvbenchli2023videommetempcompasszhou2025mlvuwu2024longvideobench and six reasoning-focused rasheed2025videomathqahu2025video_mmmummvu2024vsibenchminerva2024deng2025scivideobench datasets. The rightmost columns show the overall average. The dark blue vertical line separates the generic, reasoning, and average columns. Darker colors indicate higher scores, with the best results in bold and the second-best underlined for all benchmarks. Video-R2 achieves the best TAC on 8 out of 11 benchmarks and the best VAS on all 11 benchmarks compared with previous reasoning models. Overall, the proposed temporal alignment and consistency gating improve both logical coherence (TAC) and perceptual focus (VAS) in video reasoning multimodal models.
• Figure A1: TAC comparison Example 1.