ViRectify: A Challenging Benchmark for Video Reasoning Correction with Multimodal Large Language Models

TL;DR

ViRectify introduces a large-scale, multimodal benchmark to systematically evaluate and improve how models identify and correct video-based reasoning errors. It combines AI-assisted data creation, human verification, and a two-stage trajectory-aware correction framework that uses graph-based error modeling and temporal, video-grounded rewards. Across 16 MLLMs, the benchmark reveals substantial gaps in correction ability, with even the strongest models achieving only around 30% correction accuracy, and shows that grounding corrections in sustained video evidence and explicit error propagation tracing yields meaningful gains. The work also demonstrates the potential of reflection learning to further bolster reasoning-aware, video-grounded performance and provides a comprehensive resource for future research in robust video reasoning.

Abstract

As multimodal large language models (MLLMs) frequently exhibit errors in complex video reasoning scenarios, correcting these errors is critical for uncovering their weaknesses and improving performance. However, existing benchmarks lack systematic evaluation of MLLMs' ability to identify and correct these video reasoning errors. To bridge this gap, we propose ViRectify, a comprehensive benchmark to evaluate their fine-grained correction capability. Through an AI-assisted annotation pipeline with human verification, we construct a dataset of over 30K instances spanning dynamic perception, scientific reasoning, and embodied decision-making domains. In ViRectify, we challenge MLLMs to perform step-wise error identification and generate rationales with key video evidence grounding. In addition, we further propose the trajectory evidence-driven correction framework, comprising step-wise error trajectory and reward modeling on visual evidence-grounded correction. It encourages the model to explicitly concentrate on error propagation and key timestamps for correction. Extensive evaluation across 16 advanced MLLMs demonstrates that our ViRectify serves as a challenging testbed, where GPT-5 achieves only 31.94% correction accuracy. Our framework enables a Qwen2.5-VL-7B to consistently outperform the variants of 72B on ViRectify, showing the effectiveness of our approach. Further analysis uncovers systematic asymmetries in error correction across models, and our dataset is also a valuable data resource to perform reflection learning. We believe ViRectify provides a new direction for comprehensively evaluating the advanced MLLMs in video reasoning.

Figures (12)

• Figure 1: An example from our ViRectify benchmark. Green denotes reasoning consistent with the video evidence. Red marks steps that are incorrect or unsupported.
• Figure 2: Construction process of ViRectify. We adopt two strategies to collect erroneous solutions: (1) injecting errors with a proprietary model, and (2) collecting naturally occurring reasoning mistakes from smaller models. Then we annotate the erroneous reasoning chains with step-level error identifications and rationales grounded in video evidence.
• Figure 3: Domain distribution of ViRectify.
• Figure 4: Performance across different error types.
• Figure 5: Case study of Qwen2.5-VL-72B and our framework.