Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Select Less, Reason More: Prioritizing Evidence Purity for Video Reasoning

Xuchen Li, Xuzhao Li, Shiyu Hu, Kaiqi Huang

TL;DR

This work tackles the challenge of long-form video reasoning by addressing information dilution from static frame sampling and the lack of principled evidence-purity enforcement. It introduces the EARL framework, an evidence-aware reinforcement learning approach that turns Video LLMs into active interrogators of evidence, combining dynamic key-frame selection with localized re-sampling to obtain fine-grained temporal detail. A two-stage training pipeline—operation-aware supervised fine-tuning (SFT) followed by EARL with a multi-component reward system (including IoU-based relevance and correctness rewards and a dynamic reward scheduler)—enforces high-purity visual evidence and selective context. Experiments on five demanding benchmarks show state-of-the-art open-source performance, validating the effectiveness of evidence purity as a core objective for scalable and accurate video reasoning in LLMs. The approach demonstrates that intelligently filtering and refining visual context yields substantial gains over larger fixed-context baselines, with practical implications for efficient, high-quality long-form video understanding.

Abstract

Long-form video reasoning remains a major challenge for Video Large Language Models (Video LLMs), as static uniform frame sampling leads to information dilution and obscures critical evidence. Furthermore, existing pixel-space video reasoning agents, which are designed to actively interact with the video to acquire new visual information, remain suboptimal due to their lack of rigorous reward mechanisms to enforce evidence purity and their inability to perform temporal information supplementation beyond pre-sampled frames. To address this critical gap, we propose a novel evidence-prioritized adaptive framework built upon our core philosophy: "Select Less, Reason More." Our core contribution is the evidence-aware reinforcement learning (EARL) framework, which transforms the model into an active interrogator of evidence. EARL is precisely engineered to dynamically select the most relevant frames and, crucially, to perform localized re-sampling around the selected key frames to access fine-grained temporal detail. Extensive experiments on five demanding video reasoning benchmarks demonstrate that our EARL-trained model achieves new state-of-the-art among open-source Video LLMs, simultaneously learning an effective and high-purity visual evidence selection policy. Impressively, our 7B model achieves 59.8% on LongVideoBench, 69.0% on MVBench and 64.9% on VideoMME. These results highlight the importance of prioritizing evidence purity and the effectiveness of our framework.

Select Less, Reason More: Prioritizing Evidence Purity for Video Reasoning

TL;DR

This work tackles the challenge of long-form video reasoning by addressing information dilution from static frame sampling and the lack of principled evidence-purity enforcement. It introduces the EARL framework, an evidence-aware reinforcement learning approach that turns Video LLMs into active interrogators of evidence, combining dynamic key-frame selection with localized re-sampling to obtain fine-grained temporal detail. A two-stage training pipeline—operation-aware supervised fine-tuning (SFT) followed by EARL with a multi-component reward system (including IoU-based relevance and correctness rewards and a dynamic reward scheduler)—enforces high-purity visual evidence and selective context. Experiments on five demanding benchmarks show state-of-the-art open-source performance, validating the effectiveness of evidence purity as a core objective for scalable and accurate video reasoning in LLMs. The approach demonstrates that intelligently filtering and refining visual context yields substantial gains over larger fixed-context baselines, with practical implications for efficient, high-quality long-form video understanding.

Abstract

Long-form video reasoning remains a major challenge for Video Large Language Models (Video LLMs), as static uniform frame sampling leads to information dilution and obscures critical evidence. Furthermore, existing pixel-space video reasoning agents, which are designed to actively interact with the video to acquire new visual information, remain suboptimal due to their lack of rigorous reward mechanisms to enforce evidence purity and their inability to perform temporal information supplementation beyond pre-sampled frames. To address this critical gap, we propose a novel evidence-prioritized adaptive framework built upon our core philosophy: "Select Less, Reason More." Our core contribution is the evidence-aware reinforcement learning (EARL) framework, which transforms the model into an active interrogator of evidence. EARL is precisely engineered to dynamically select the most relevant frames and, crucially, to perform localized re-sampling around the selected key frames to access fine-grained temporal detail. Extensive experiments on five demanding video reasoning benchmarks demonstrate that our EARL-trained model achieves new state-of-the-art among open-source Video LLMs, simultaneously learning an effective and high-purity visual evidence selection policy. Impressively, our 7B model achieves 59.8% on LongVideoBench, 69.0% on MVBench and 64.9% on VideoMME. These results highlight the importance of prioritizing evidence purity and the effectiveness of our framework.
Paper Structure (20 sections, 9 equations, 5 figures, 3 tables)

This paper contains 20 sections, 9 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Textual-space Video Reasoning
  4. Pixel-space Video Reasoning
  5. Problem Formulation
  6. Method
  7. Operation-Aware Supervised Fine-Tuning
  8. Evidence-Aware Reinforcement Learning
  9. Key-frame Annotation
  10. Reward Function Design
  11. Dynamic Adjustment of Reward Sensitivity
  12. Experiments
  13. Setups
  14. Main Results
  15. Ablation Study
  16. ...and 5 more sections

Figures (5)

  • Figure 1: The core motivation and mechanism of our evidence-prioritized adaptive pixel-space video reasoning framework. Existing approaches are limited by three factors ($\times$): 1) Static uniformly sampling dilutes the context with redundant frames; 2) Frame selection within pre-sampled frames restricts access to necessary fine-grained temporal detail; and 3) Selection with new sampled frames without key-frame label supervision fails to enforce evidence purity, potentially leading to sampling in irrelevant areas. Our proposed method ($\checkmark$) overcomes these limitations by integrating frame selection with localized re-sampling to acquire fine-grained temporal detail, and applying key-frame label supervision (via the IoU-based reward) to ensure high evidence purity.
  • Figure 2: Overview of the Evidence-Aware Reinforcement Learning (EARL) framework. In the multi-round generation process, the model can attend to select frames adaptively and integrate the result of key-frame based localized re-sampling to form a multimodal CoT.
  • Figure 3: Our EARL framework ensures accuracy by actively interrogating evidence. For the complex counting question, the model uses a CoT step to identify the information gap, then calls the select_frames tool for key-frame #24. This action triggers localized re-sampling, providing a high-purity, fine-grained visual context. Utilizing this refined evidence stream, the model correctly answers.
  • Figure 4: Ablation study on the effectiveness of EARL.
  • Figure 5: Ablation study on the effectiveness of relevance reward.