Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Video Evidence to Reasoning Efficient Video Understanding via Explicit Evidence Grounding

Yanxiang Huang, Guohua Gao, Zhaoyang Wei, Jianyuan Ni

TL;DR

This work tackles the fundamental trade-off between perceptual grounding accuracy and reasoning efficiency in video LVLMs, which often leads to hallucinations when relying on ungrounded chain-of-thought. It introduces the Chain of Evidence (CoE) framework, decoupling perception and reasoning through a lightweight Evidence Grounding Module (EGM) and a structured CoE Reasoning Protocol that anchors deductions to explicit temporal evidence. A large-scale CoE-Instruct dataset (164k samples) supports a decoupled training regime with grounding and reasoning losses, plus reinforcement learning via an evidence-aware reward (GRPO) to align perception with reasoning. Empirical results across five benchmarks show state-of-the-art accuracy and substantial reductions in token usage and latency for open-source models, validating CoE as a practical paradigm for reliable and efficient video understanding.

Abstract

Large Vision-Language Models (LVLMs) face a fundamental dilemma in video reasoning: they are caught between the prohibitive computational costs of verbose reasoning and the hallucination risks of efficient, ungrounded approaches. To resolve this, we introduce the Chain of Evidence (CoE), a novel framework that architecturally decouples and co-optimizes perceptual grounding and reasoning efficiency. CoE incorporates two core innovations: (1) A lightweight Evidence Grounding Module (EGM) that acts as a query-guided filter, dynamically identifying and extracting a compact set of high-fidelity visual evidence; and (2) An Evidence-Anchoring Protocol optimized via Reinforcement Learning. Crucially, we design a composite reward mechanism that enforces process alignment, compelling the model to strictly reference identified temporal anchors during deduction, thereby mitigating hallucinations. To enable this, we construct CoE-Instruct, a large-scale dataset (164k samples) featuring a novel dual-annotation schema for separate perception and reasoning supervision. Extensive experiments on five benchmarks, including Video-MME, MVBench, and VSI-Bench, demonstrate that CoE-enhanced models establish a new state-of-the-art. They significantly outperform existing methods in accuracy, proving CoE to be a powerful and practical paradigm for reliable video understanding.

Video Evidence to Reasoning Efficient Video Understanding via Explicit Evidence Grounding

TL;DR

This work tackles the fundamental trade-off between perceptual grounding accuracy and reasoning efficiency in video LVLMs, which often leads to hallucinations when relying on ungrounded chain-of-thought. It introduces the Chain of Evidence (CoE) framework, decoupling perception and reasoning through a lightweight Evidence Grounding Module (EGM) and a structured CoE Reasoning Protocol that anchors deductions to explicit temporal evidence. A large-scale CoE-Instruct dataset (164k samples) supports a decoupled training regime with grounding and reasoning losses, plus reinforcement learning via an evidence-aware reward (GRPO) to align perception with reasoning. Empirical results across five benchmarks show state-of-the-art accuracy and substantial reductions in token usage and latency for open-source models, validating CoE as a practical paradigm for reliable and efficient video understanding.

Abstract

Large Vision-Language Models (LVLMs) face a fundamental dilemma in video reasoning: they are caught between the prohibitive computational costs of verbose reasoning and the hallucination risks of efficient, ungrounded approaches. To resolve this, we introduce the Chain of Evidence (CoE), a novel framework that architecturally decouples and co-optimizes perceptual grounding and reasoning efficiency. CoE incorporates two core innovations: (1) A lightweight Evidence Grounding Module (EGM) that acts as a query-guided filter, dynamically identifying and extracting a compact set of high-fidelity visual evidence; and (2) An Evidence-Anchoring Protocol optimized via Reinforcement Learning. Crucially, we design a composite reward mechanism that enforces process alignment, compelling the model to strictly reference identified temporal anchors during deduction, thereby mitigating hallucinations. To enable this, we construct CoE-Instruct, a large-scale dataset (164k samples) featuring a novel dual-annotation schema for separate perception and reasoning supervision. Extensive experiments on five benchmarks, including Video-MME, MVBench, and VSI-Bench, demonstrate that CoE-enhanced models establish a new state-of-the-art. They significantly outperform existing methods in accuracy, proving CoE to be a powerful and practical paradigm for reliable video understanding.
Paper Structure (28 sections, 7 equations, 2 figures, 5 tables)

This paper contains 28 sections, 7 equations, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. The Chain of Evidence Framework
  3. Architectural Design: A Hierarchical Approach
  4. The Evidence Grounding Module (EGM)
  5. The CoE Reasoning Protocol
  6. Decoupled Joint Training Strategy for SFT
  7. Grounding Loss ($L_{\text{grounding}}$)
  8. Reasoning Loss ($L_{\text{reasoning}}$)
  9. Reinforcement Learning from Evidence Feedback
  10. Reward Modeling
  11. Optimization
  12. Structured Inference and Interpretability
  13. Dataset Construction
  14. CoE-Instruct: A Multi-Stage Training Dataset
  15. Phase 1: CoE-Instruct-SFT for Foundational Skill Acquisition.
  16. ...and 13 more sections

Figures (2)

  • Figure 1: The EGM acts as a query-guided filter between the vision encoder and the language decoder, enabling a hierarchical processing of visual information.
  • Figure 2: The framework of CoE, where the EGM first extracts compact visual evidence based on the query, enabling the LLM to perform efficient and grounded reasoning.