Video-KTR: Reinforcing Video Reasoning via Key Token Attribution

TL;DR

Video-KTR addresses the challenge of video reasoning by replacing coarse sequence-level RL rewards with token-level, modality-aware credit assignment. The method introduces three attribution signals—visual-aware via counterfactual masking, temporal-aware via frame shuffling, and entropy-aware tokens—then selectively updates the most informative tokens using a GRPO-based policy objective. This leads to state-of-the-art or competitive results across five benchmarks, including 42.7% on Video-Holmes (comparable to GPT-4o) and strong performance on knowledge-intensive tasks, while ablations confirm the complementary roles of the three signals. Overall, Video-KTR offers a simple, interpretable, and robust extension to RL for complex video reasoning with large multimodal models.

Abstract

Reinforcement learning (RL) has shown strong potential for enhancing reasoning in multimodal large language models, yet existing video reasoning methods often rely on coarse sequence-level rewards or single-factor token selection, neglecting fine-grained links among visual inputs, temporal dynamics, and linguistic outputs, limiting both accuracy and interpretability. We propose Video-KTR, a modality-aware policy shaping framework that performs selective, token-level RL by combining three attribution signals: (1) visual-aware tokens identified via counterfactual masking to reveal perceptual dependence; (2) temporal-aware tokens detected through frame shuffling to expose temporal sensitivity; and (3) high-entropy tokens signaling predictive uncertainty. By reinforcing only these key tokens, Video-KTR focuses learning on semantically informative, modality-sensitive content while filtering out low-value tokens. Across five challenging benchmarks, Video-KTR achieves state-of-the-art or highly competitive results, achieving 42.7\% on Video-Holmes (surpassing GPT-4o) with consistent gains on both reasoning and general video understanding tasks. Ablation studies verify the complementary roles of the attribution signals and the robustness of targeted token-level updates. Overall, Video-KTR improves accuracy and interpretability, offering a simple, drop-in extension to RL for complex video reasoning. Our code and models are available at https://github.com/zywang0104/Video-KTR.

Figures (14)

• Figure 1: Overview of the Video-KTR framework. The model identifies key tokens based on entropy, visual, and temporal signals, and updates only those tokens during reinforcement learning.
• Figure 2: An illustration of critical tokens identified by each attribution strategy.
• Figure 3: Performance comparison on (a) video reasoning benchmarks and (b) detailed subtasks of the Video-Holmes benchmark. Subtask abbreviations: SR = Social Reasoning, IMC = Intention & Motive Chaining, TCI = Temporal Causal Inference, TA = Timeline Analysis, MHR = Multimodal Hint Reasoning, PAR = Physical Anomaly Reasoning, CTI = Core Theme Inference.
• Figure 4: POS ratio across selection strategies.
• Figure 5: Distribution of updated tokens.