Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Structured Over Scale: Learning Spatial Reasoning from Educational Video

Bishoy Galoaa, Xiangyu Bai, Sarah Ostadabbas

TL;DR

This work addresses the failure of vision-language models to perform robust spatial reasoning by leveraging the structured pedagogy of children's educational content. It introduces DoraVQA, a dataset of $5{,}344$ QA pairs tied to $96$ Dora the Explorer episodes and aligned with a context-question-pause-answer format, and fine-tunes Qwen2-VL and Qwen3-VL using Group Relative Policy Optimization on about $38$ hours of content. The approach yields substantial improvements on DoraVQA and achieves state-of-the-art $86.16\%$ on CVBench, with strong transfer to Video-MME and NExT-QA, demonstrating that content structure can compensate for scale. These results support the core claim that structured supervision can endow VLMs with more robust spatial reasoning across diverse tasks and benchmarks, suggesting a broader role for pedagogical content in multimodal learning.

Abstract

Vision-language models (VLMs) demonstrate impressive performance on standard video understanding benchmarks yet fail systematically on simple reasoning tasks that preschool children can solve, including counting, spatial reasoning, and compositional understanding. We hypothesize that the pedagogically-structured content of educational videos provides an ideal training signal for improving these capabilities. We introduce DoraVQA, a dataset of 5,344 question-answer pairs automatically extracted from 8 seasons of Dora the Explorer with precise timestamp alignment. Each episode follows a consistent \textit{context-question-pause-answer} structure that creates a self-contained learning environment analogous to interactive tutoring. We fine-tune both Qwen2 and Qwen3 using Group Relative Policy Optimization (GRPO), leveraging the clear correctness signals and structured reasoning traces inherent in educational content. Despite training exclusively on 38 hours of children's educational videos, our approach achieves improvements of 8-14 points on DoraVQA and state-of-the-art 86.16\% on CVBench, with strong transfer to Video-MME and NExT-QA, demonstrating effective generalization from narrow pedagogical content to broad multimodal understanding. Through cross-domain benchmarks, we show that VLMs can perform tasks that require robust reasoning learned from structured educational content, suggesting that content structure matters as much as content scale.

Structured Over Scale: Learning Spatial Reasoning from Educational Video

TL;DR

This work addresses the failure of vision-language models to perform robust spatial reasoning by leveraging the structured pedagogy of children's educational content. It introduces DoraVQA, a dataset of QA pairs tied to Dora the Explorer episodes and aligned with a context-question-pause-answer format, and fine-tunes Qwen2-VL and Qwen3-VL using Group Relative Policy Optimization on about hours of content. The approach yields substantial improvements on DoraVQA and achieves state-of-the-art on CVBench, with strong transfer to Video-MME and NExT-QA, demonstrating that content structure can compensate for scale. These results support the core claim that structured supervision can endow VLMs with more robust spatial reasoning across diverse tasks and benchmarks, suggesting a broader role for pedagogical content in multimodal learning.

Abstract

Vision-language models (VLMs) demonstrate impressive performance on standard video understanding benchmarks yet fail systematically on simple reasoning tasks that preschool children can solve, including counting, spatial reasoning, and compositional understanding. We hypothesize that the pedagogically-structured content of educational videos provides an ideal training signal for improving these capabilities. We introduce DoraVQA, a dataset of 5,344 question-answer pairs automatically extracted from 8 seasons of Dora the Explorer with precise timestamp alignment. Each episode follows a consistent \textit{context-question-pause-answer} structure that creates a self-contained learning environment analogous to interactive tutoring. We fine-tune both Qwen2 and Qwen3 using Group Relative Policy Optimization (GRPO), leveraging the clear correctness signals and structured reasoning traces inherent in educational content. Despite training exclusively on 38 hours of children's educational videos, our approach achieves improvements of 8-14 points on DoraVQA and state-of-the-art 86.16\% on CVBench, with strong transfer to Video-MME and NExT-QA, demonstrating effective generalization from narrow pedagogical content to broad multimodal understanding. Through cross-domain benchmarks, we show that VLMs can perform tasks that require robust reasoning learned from structured educational content, suggesting that content structure matters as much as content scale.
Paper Structure (16 sections, 3 equations, 4 figures, 6 tables, 1 algorithm)

This paper contains 16 sections, 3 equations, 4 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. DoraVQA: Structure and Formatting
  5. Group Relative Policy Optimization
  6. Training
  7. Experiments and Results
  8. Datasets
  9. Main Results on DoraVQA
  10. Cross-Benchmark Transfer
  11. Ablation Studies
  12. Conclusion
  13. Impact Statements
  14. Hyperparameter Sweeps
  15. Additional Qualitative Results
  16. ...and 1 more sections

Figures (4)

  • Figure 1: Our DoraVQA dataset composition across three key dimensions. (a) Reasoning categories divide into spatial tasks (60.6%), including object selection (35.4%), spatial location (42.2%), and navigation (22.4%), and non-spatial tasks (38.6%), encompassing language (18.9%), counting (6.5%), knowledge recall (42.6%), and problem solving (32.0%). (b) Input modality shows balanced distribution across text-only (36.4%), visual-only (27.3%), and multimodal questions (36.3%). (c) Temporal structure reveals that most questions require immediate reasoning (78.8%), while sequential reasoning across multiple frames accounts for 23.2%.
  • Figure 2: DoraVQA pipeline overview. We extract question-answer pairs from Dora the Explorer episodes by parsing SRT transcript files with a Qwen agent, aligning timestamps to identify the show's pedagogical context-question-pause-answer structure. Each detected question is paired with its surrounding context window and the ground truth answer that follows. During training, we fine-tune Qwen2-VL and Qwen3-VL using GRPO on open-ended generation with reasoning; rewards are computed from F1 score and normalized Levenshtein distance against transcript ground truth. A Gemini agent generates multiple-choice distractors from the ground truth answers, which are human-audited for quality. During inference, the MCQ options are provided to the fine-tuned VLM for selection, creating a deliberate train-test format mismatch (open-ended $\rightarrow$ MCQ) that evaluates transferable reasoning. We report top-1 accuracy as the evaluation metric.
  • Figure 3: Qualitative comparison on challenging spatial reasoning tasks. Our GRPO-finetuned models demonstrate superior performance on: (1) Spatial location where the chocolate boat is camouflaged against the chocolate river, (2) Object selection requiring identification of distant boats in the background, where baselines correctly identify spatial position but hallucinate object properties, (3) Navigation requiring sequential recall of past events to determine the next destination, and (4) Counting of visually similar crocodiles positioned close together, requiring frame-by-frame enumeration. Baseline models (orange boxes) fail or hallucinate, while our models (blue boxes) provide accurate, concise answers. See Appendix \ref{['sec:additional_qualitative']} and supplementary material for more examples.
  • Figure 4: Additional challenging examples from DoraVQA. Left: Spatial location task where Swiper the fox is partially occluded behind Dora, requiring detection of partially visible objects. Right: Counting task requiring enumeration of 8 identical points on a key positioned closely together. Baseline models fail on both tasks, while our GRPO-finetuned models provide correct answers.