LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoning

TL;DR

LOVE-R1 tackles long video understanding by combining a slow-fast adaptive frame template with a multi-step reasoning framework. It first analyzes videos globally at low resolution and then adaptively zooms in on informative clips at high resolution, guided by a decision-zoom-answer loop trained through a three-stage post-training pipeline. The approach, including decoupled reinforcement finetuning for fine-grained zoom-in supervision, achieves state-of-the-art results on major long-video benchmarks and demonstrates strong balancing of temporal density and spatial detail. This work introduces a practical paradigm for LVLM-based LVU and emphasizes the value of high-quality long-video data and potential for longer context in future research.

Abstract

Long video understanding is still challenging for recent Large Video-Language Models (LVLMs) due to the conflict between long-form temporal understanding and detailed spatial perception. LVLMs with a uniform frame sampling mechanism, which samples frames with an equal frame size and fixed sampling rate, inevitably sacrifice either temporal clues or spatial details, resulting in suboptimal solutions. To mitigate this dilemma, we propose LOVE-R1, a model that can adaptively zoom in on a video clip. The model is first provided with densely sampled frames but in a small resolution. If some spatial details are needed, the model can zoom in on a clip of interest with a large frame resolution based on its reasoning until key visual information is obtained. The whole process is implemented as a multi-step reasoning process. To train the reasoning ability, we first finetune the model on our collected 38k high-quality CoT data and enhance it with decoupled reinforcement finetuning. As outcome rewards can not provide fine-grained process supervision, we decouple multi-step reasoning into multiple single-step reasoning and optimize the internal zoom-in ability explicitly. Experiments on long video understanding benchmarks show that our model with the slow-fast adaptive frame sampling mechanism achieves a great trade-off between sampling density and frame resolutions, and LOVE-R1 outperforms our baseline Qwen2.5-VL by an average of 3.1% points across 4 common long video understanding benchmarks.

Figures (7)

• Figure 1: Illustration of the workflow of LOVE-R1. Our model first takes densely sampled small-resolution frames from the whole video as inputs to understand the video globally. If needed, it can adaptively zoom in on a video clip to gain fine-grained spatial details. The workflow is implemented as a multi-step reasoning process.
• Figure 2: Different slow-fast video templates. Templates (a) and (b) will replace the original fast video segments with the slow videos. Template (a) treats multiple video segments as a whole video while Template (b) explicitly separates them with identifiers (<| vision_start|>, <| vision_end|>). Template (c) appends the additional slow videos at the end of the fast video without removing the corresponding fast video segments. We adopt Template (c).
• Figure 3: Illustration of decoupled reinforcement finetuning. (a) For questions without ground truth timespans, we apply the standard GRPO algorithm to optimize multi-step CoTs as a whole. (b) To provide fine-grained process rewards, we decouple multi-step reasoning into multiple single-step reasoning and optimize the single zoom-in step explicitly by appending the zoom-in prefix.
• Figure 4: Visualization of LOVE-R1 inference results. The video is taken from Video-MME (vid: edAu5_O4C54).
• Figure 5: Our CoT data construction pipeline. To ensure the data quality, we perform strict data pre-processing and post-processing by filtering out low-quality annotations and CoTs. We also use a strong reasoning model, Gemini 2.5 pro, to annotate CoT data, ensuring the content of CoTs is reasonable and high-quality.