VideoCuRL: Video Curriculum Reinforcement Learning with Orthogonal Difficulty Decomposition

TL;DR

VideoCuRL tackles the challenge of post-training video reasoning by explicitly separating difficulty into Visual-Temporal Perception Load and Cognitive Reasoning Depth, and by employing a 2D curriculum governed by a competence-aware Diagonal Wavefront. It uses training-free proxies, $D_{visual}$ and $D_{text}$, to map data onto a $K\times K$ grid and applies Dynamic Sparse KL and Structured Revisiting to stabilize RL and mitigate forgetting. The approach, built on Group Relative Policy Optimization, achieves state-of-the-art or competitive results on both reasoning-intensive benchmarks (e.g., VSI-Bench) and general video perception benchmarks, while incurring negligible overhead compared with generation-based curricula. The findings underscore that data organization, not just scale, is pivotal for scalable, robust Video-LMM post-training, with practical implications for efficient curriculum design in multimodal RL.

Abstract

Reinforcement Learning (RL) is crucial for empowering VideoLLMs with complex spatiotemporal reasoning. However, current RL paradigms predominantly rely on random data shuffling or naive curriculum strategies based on scalar difficulty metrics. We argue that scalar metrics fail to disentangle two orthogonal challenges in video understanding: Visual Temporal Perception Load and Cognitive Reasoning Depth. To address this, we propose VideoCuRL, a novel framework that decomposes difficulty into these two axes. We employ efficient, training-free proxies, optical flow and keyframe entropy for visual complexity, Calibrated Surprisal for cognitive complexity, to map data onto a 2D curriculum grid. A competence aware Diagonal Wavefront strategy then schedules training from base alignment to complex reasoning. Furthermore, we introduce Dynamic Sparse KL and Structured Revisiting to stabilize training against reward collapse and catastrophic forgetting. Extensive experiments show that VideoCuRL surpasses strong RL baselines on reasoning (+2.5 on VSI-Bench) and perception (+2.9 on VideoMME) tasks. Notably, VideoCuRL eliminates the prohibitive inference overhead of generation-based curricula, offering a scalable solution for robust video post-training.

Figures (4)

• Figure 1: Multidimensional performance comparison. (Left) Comparison with baselines. (Right) Comparison with other open-source models.
• Figure 2: Overview of VideoCuRL. VideoCuRL disentangles video difficulty into orthogonal visual-temporal ($D_{visual}$) and cognitive ($D_{text}$) dimensions. A competence-aware Diagonal Wavefront strategy schedules training across a 2D grid, supported by Dynamic Sparse KL and Structured Revisiting to stabilize RL and prevent forgetting.
• Figure 3: Visual Complexity Estimation. Perceptual load is quantified via dual-stream processing: Motion Intensity ($\phi_{flow}$) from optical flow and Information Density ($\phi_{ent}$) from frame-difference entropy. Scores are fused and quantile-normalized to ensure a balanced difficulty distribution.
• Figure 4: Quantitative Cross-Validation of Difficulty Proxies. We analyze the correlation between our training-free difficulty metrics and model accuracy on the training set. (a-b) Individual visual proxies ($\phi_{flow}$, $\phi_{ent}$) exhibit high variance, indicating that physical features alone are noisy predictors of difficulty. (c) The fused Visual Complexity ($D_{visual}$) smooths these fluctuations, providing a robust estimator of perceptual load. (d) Cognitive Complexity ($D_{text}$) reveals the nonlinear relationship between logical challenge and model performance.