Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

VISTA: Mitigating Semantic Inertia in Video-LLMs via Training-Free Dynamic Chain-of-Thought Routing

Hongbo Jin, Jiayu Ding, Siyi Xie, Guibo Luo, Ge Li

TL;DR

This work identifies Semantic Inertia, a misalignment where Video-LLMs underutilize valid visual evidence due to strong language priors. It introduces VISTA, a training-free framework that aligns perception and reasoning via Dynamic Inference Routing, Explicit Visual Anchoring, and Latent Reasoning Consensus to produce evidence-grounded deductions. By materializing visual cues as textual anchors and enforcing multi-path consensus, VISTA mitigates hallucinations and achieves strong results on multiple video QA benchmarks, rivaling larger proprietary models without parameter updates. The approach offers a scalable path toward robust System 2 video reasoning with practical implications for reliable multimodal reasoning systems.

Abstract

Recent advancements in Large Language Models have successfully transitioned towards System 2 reasoning, yet applying these paradigms to video understanding remains challenging. While prevailing research attributes failures in Video-LLMs to perceptual limitations, our empirical analysis reveals a cognitive misalignment termed Semantic Inertia, where models suppress valid visual evidence in favor of dominant language priors. To rectify this, we propose VISTA, a training-free framework designed to align perception with logical deduction. By dynamically routing inference paths and materializing implicit visual features into explicit textual anchors, our approach effectively counterbalances the influence of parametric knowledge. Furthermore, we incorporate a Latent Reasoning Consensus mechanism to mitigate stochastic hallucinations. VISTA showed outstanding results on a wide range of benchmarks, and outperforms its base model by 9.3% on Egochema and 5.6% on VideoEspresso, rivalling or even surpassing larger and proprietary models. Our codebase will be publicly available soon.

VISTA: Mitigating Semantic Inertia in Video-LLMs via Training-Free Dynamic Chain-of-Thought Routing

TL;DR

This work identifies Semantic Inertia, a misalignment where Video-LLMs underutilize valid visual evidence due to strong language priors. It introduces VISTA, a training-free framework that aligns perception and reasoning via Dynamic Inference Routing, Explicit Visual Anchoring, and Latent Reasoning Consensus to produce evidence-grounded deductions. By materializing visual cues as textual anchors and enforcing multi-path consensus, VISTA mitigates hallucinations and achieves strong results on multiple video QA benchmarks, rivaling larger proprietary models without parameter updates. The approach offers a scalable path toward robust System 2 video reasoning with practical implications for reliable multimodal reasoning systems.

Abstract

Recent advancements in Large Language Models have successfully transitioned towards System 2 reasoning, yet applying these paradigms to video understanding remains challenging. While prevailing research attributes failures in Video-LLMs to perceptual limitations, our empirical analysis reveals a cognitive misalignment termed Semantic Inertia, where models suppress valid visual evidence in favor of dominant language priors. To rectify this, we propose VISTA, a training-free framework designed to align perception with logical deduction. By dynamically routing inference paths and materializing implicit visual features into explicit textual anchors, our approach effectively counterbalances the influence of parametric knowledge. Furthermore, we incorporate a Latent Reasoning Consensus mechanism to mitigate stochastic hallucinations. VISTA showed outstanding results on a wide range of benchmarks, and outperforms its base model by 9.3% on Egochema and 5.6% on VideoEspresso, rivalling or even surpassing larger and proprietary models. Our codebase will be publicly available soon.
Paper Structure (25 sections, 7 equations, 5 figures, 5 tables)

This paper contains 25 sections, 7 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Pilot Experiments
  3. The Perception-Reasoning Gap
  4. Semantic Inertia Suppression
  5. Method
  6. Dynamic Inference Routing
  7. Explicit Visual Anchoring
  8. Latent Reasoning Consensus
  9. Experiment
  10. Settings
  11. Main Performance on Video QA
  12. Ablation Studies
  13. Related Work
  14. Conclusion
  15. Case Study
  16. ...and 10 more sections

Figures (5)

  • Figure 1: Performance Discrepancies across tasks.
  • Figure 2: Overview of the VISTA framework. The pipeline begins with Dynamic Inference Routing, where the Taxonomy Routing module references the external Taxonomy feature table to identify the query type (e.g., causal reasoning). For complex queries, the Visual Materializer translates visual cues into a Structured Visual Summary. This output supports the Latent Reasoning Consensus stage, which filters hallucinations through multi-path sampling and similarity-based clustering. Finally, the selected reasoning path is fused into Aggregated Comprehensive Evidence, guiding the Video Large Language Model to generate the final response.
  • Figure 3: Verification methods overview. We show the implementation and core differences between four different verification mechanisms.
  • Figure 4: Trend of inference performance with number of samples
  • Figure 5: A typical case to illustrate the superiority of VISTA.