TIR-Flow: Active Video Search and Reasoning with Frozen VLMs

TL;DR

TIR-Flow reframes video reasoning as an active search problem that leverages frozen vision-language models without fine-tuning. By integrating Hypothesis-Driven Decomposition, Hierarchical Active Perception, and an Evidence Blackboard with Arbitration, it enables a Plan-Look-Verify loop that grounds reasoning in verifiable visual evidence, addressing perceptual bottlenecks and long-horizon inference. Empirical results across seven benchmarks show a robust average improvement of 5.9%, with peaks up to 10.5% on EgoSchema, outperforming several baselines while avoiding additional data or parameter updates. This approach demonstrates that equipping VLMs with active perception at inference time can achieve scalable, reliable video reasoning with reduced training costs.

Abstract

While Large Video-Language Models (Video-LLMs) have achieved remarkable progress in perception, their reasoning capabilities remain a bottleneck. Existing solutions typically resort to a heavy "data engineering" paradigm-synthesizing large-scale Chain-of-Thought (CoT) datasets followed by Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL). This pipeline primarily optimizes probability sampling efficiency and aligns output distributions, but fails to activate the intrinsic intelligence required for dynamic visual exploration. In this work, we propose TIR-Flow, a novel framework that shifts the paradigm from passive processing to active video searching and reasoning without additional data or parameter updating. Concretely, our framework operates through three synergistic modules: HDD decomposes complex queries into a set of verifiable sub-tasks; HAP actively directs visual attention to gather high-resolution evidence for hypothesis validation; EBA maintains a persistent workspace to accumulate and update the discovered clues for logical reasoning. Extensive experiments on seven benchmarks demonstrate that TIR-Flow significantly outperforms recent strong baselines, delivering an average performance boost of 5.9%, with gains reaching 10.5% on Egoschema. Our analysis confirms that empowering frozen VLMs with System-2-like active perception is a scalable path toward solving long-horizon video reasoning.

Figures (5)

• Figure 1: Unlocking reasoning potential. TIR-Flow consistently boosts the zero-shot performance of (a) LLaVA-Video-7B and (b) Qwen2.5-VL-7B across seven benchmarks, improving both perception and reasoning capabilities.
• Figure 2: Comparison of Reasoning Trajectories between Passive CoT and TIR-Flow Paradigm. Without an explicit framework to guide the model in extracting and verifying visual information, it is challenging for the system to perform logical deductions grounded in the intrinsic content of the video.
• Figure 3: Overall Architecture of the TIR-Flow. First, the user question is decomposed by an LLM into structured sub-queries. Next, hierarchical active perception collects targeted evidence via coarse-to-fine spatial crops. Finally, an evidence blackboard arbitrates and scores the evidence, and the prediction is fused with the query to generate the final response.
• Figure 4: Two Stage Grid Crop Pipeline: Similarity Based Keyframe Scouting ($k=5$) Followed by Grid-Pyramid Patch Retrieval.
• Figure 5: Impact of Diverse Sample Size ($k$) on EgoSchema and VideoMME.