A Very Big Video Reasoning Suite

TL;DR

VBVR introduces the Very Big Video Reasoning suite, addressing the lack of large-scale video reasoning data and verifiable evaluation by 1) delivering the VBVR-Dataset (≈2.0M images and ≈1.0M video clips across 200 tasks) and 2) the VBVR-Bench with rule-based scoring and human-alignment validation. The framework employs a cognitive-architecture taxonomy (Abstraction, Knowledge, Perception, Spatiality, Transformation) implemented as parameterized task generators, enabling scalable, diverse data generation and automated quality control. A systematic scaling study using Wan2.2 demonstrates early signs of generalization with increased data but reveals persistent gaps to human performance and ID–OOD transfer limitations, highlighting the need for controllability-centric evaluation and architectural innovations. VBVR thus provides a scalable, extensible foundation for advancing generalizable video reasoning, with public data, benchmarks, and models to accelerate progress across academia and industry.

Abstract

Rapid progress in video models has largely focused on visual quality, leaving their reasoning capabilities underexplored. Video reasoning grounds intelligence in spatiotemporally consistent visual environments that go beyond what text can naturally capture, enabling intuitive reasoning over spatiotemporal structure such as continuity, interaction, and causality. However, systematically studying video reasoning and its scaling behavior is hindered by the lack of large-scale training data. To address this gap, we introduce the Very Big Video Reasoning (VBVR) Dataset, an unprecedentedly large-scale resource spanning 200 curated reasoning tasks following a principled taxonomy and over one million video clips, approximately three orders of magnitude larger than existing datasets. We further present VBVR-Bench, a verifiable evaluation framework that moves beyond model-based judging by incorporating rule-based, human-aligned scorers, enabling reproducible and interpretable diagnosis of video reasoning capabilities. Leveraging the VBVR suite, we conduct one of the first large-scale scaling studies of video reasoning and observe early signs of emergent generalization to unseen reasoning tasks. Together, VBVR lays a foundation for the next stage of research in generalizable video reasoning. The data, benchmark toolkit, and models are publicly available at https://video-reason.com/ .

Figures (14)

• Figure 1: Overview of VBVR. Left: the grid shows representative tasks spanning our cognitive architecture, which are color-coded according to their underlying capability: Spatiality, Transformation, Knowledge, Abstraction, and Perception. At the center of the grids, we visualize the scale comparison between VBVR (2.015M samples) and nine other datasets combined (12.8K samples): the sizes of the circles are drawn to scale. Top-right: scaling behavior on in-domain and out-of-domain evaluations. Bottom-right: benchmark performance across five cognitive capabilities.
• Figure 2: Sample task instances generated from the VBVR parameterized task suite, organized by five cognitive faculties. Each sequence illustrates the structured reasoning process required to reach a valid solution. Tasks are implemented as deterministic generators supporting scalable instance variation while preserving visual clarity and video dependency. Each row corresponds to a faculty defined in Section 3.1: abstract cognitive constructs are instantiated as executable, verifiable video-based reasoning tasks.
• Figure 3: Task designs grounded in cognitive architecture are implemented as parameterized generators, then executed at scale via distributed Lambda workers writing to centralized S3 storage.
• Figure 4: Human alignment analysis for VBVR-Bench. Our experiments show that VBVR-Bench evaluations in all splits closely match human perceptions. In each plot, a dot represents the human preference win ratio (horizontal axis) and VBVR-Bench evaluation win ratio (vertical axis) for a particular video generation model. We linearly fit a straight line to visualize the correlation, and calculate the Spearman's correlation coefficient ($\rho$) for each dimension.
• Figure 5: Residualized capability correlation among five faculties across 9 models (Pearson $\rho$). We regress out a model-level general factor (overall strength) to highlight structural dependencies and inter-relations.