Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TinyLLaVA-Video-R1: Towards Smaller LMMs for Video Reasoning

Xingjian Zhang, Siwei Wen, Wenjun Wu, Lei Huang

TL;DR

The paper tackles the challenge of enabling video reasoning in small-scale, resource-constrained models. It introduces TinyLLaVA-Video-R1, a ≤4B parameter model built on TinyLLaVA-Video and trained with Group Relative Policy Optimization on general Video-QA data, augmented with a think/answer prompt format to reveal reasoning traces. Results show substantial gains in reasoning and the emergence of aha moments even in a small model, with ablations highlighting the importance of cold-start data and reward design. The work provides practical guidance for future research in small-scale video reasoning and releases data and methods to support reproducibility and further exploration.

Abstract

Recently, improving the reasoning ability of large multimodal models (LMMs) through reinforcement learning has made great progress. However, most existing works are based on highly reasoning-intensive datasets such as mathematics and code, and researchers generally choose large-scale models as the foundation. We argue that exploring small-scale models' reasoning capabilities remains valuable for researchers with limited computational resources. Moreover, enabling models to explain their reasoning processes on general question-answering datasets is equally meaningful. Therefore, we present the small-scale video reasoning model TinyLLaVA-Video-R1. Based on TinyLLaVA-Video, a traceably trained video understanding model with no more than 4B parameters, it not only demonstrates significantly improved reasoning and thinking capabilities after using reinforcement learning on general Video-QA datasets, but also exhibits the emergent characteristic of "aha moments". Furthermore, we share a series of experimental findings, aiming to provide practical insights for future exploration of video reasoning (thinking) abilities in small-scale models. It is available at https://github.com/ZhangXJ199/TinyLLaVA-Video-R1.

TinyLLaVA-Video-R1: Towards Smaller LMMs for Video Reasoning

TL;DR

The paper tackles the challenge of enabling video reasoning in small-scale, resource-constrained models. It introduces TinyLLaVA-Video-R1, a ≤4B parameter model built on TinyLLaVA-Video and trained with Group Relative Policy Optimization on general Video-QA data, augmented with a think/answer prompt format to reveal reasoning traces. Results show substantial gains in reasoning and the emergence of aha moments even in a small model, with ablations highlighting the importance of cold-start data and reward design. The work provides practical guidance for future research in small-scale video reasoning and releases data and methods to support reproducibility and further exploration.

Abstract

Recently, improving the reasoning ability of large multimodal models (LMMs) through reinforcement learning has made great progress. However, most existing works are based on highly reasoning-intensive datasets such as mathematics and code, and researchers generally choose large-scale models as the foundation. We argue that exploring small-scale models' reasoning capabilities remains valuable for researchers with limited computational resources. Moreover, enabling models to explain their reasoning processes on general question-answering datasets is equally meaningful. Therefore, we present the small-scale video reasoning model TinyLLaVA-Video-R1. Based on TinyLLaVA-Video, a traceably trained video understanding model with no more than 4B parameters, it not only demonstrates significantly improved reasoning and thinking capabilities after using reinforcement learning on general Video-QA datasets, but also exhibits the emergent characteristic of "aha moments". Furthermore, we share a series of experimental findings, aiming to provide practical insights for future exploration of video reasoning (thinking) abilities in small-scale models. It is available at https://github.com/ZhangXJ199/TinyLLaVA-Video-R1.

Paper Structure

This paper contains 20 sections, 4 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Large Language Reasoning Models.
  4. Multimodal Reasoning Models.
  5. Methods
  6. TinyLLaVA-Video
  7. Group Relative Policy Optimization (GRPO)
  8. Training Data and Template.
  9. Reward Rules.
  10. Format reward.
  11. Accuracy reward.
  12. Experiments
  13. Experimental Settings
  14. Main Results and Aha Moment
  15. Ablation Study
  16. ...and 5 more sections

Figures (6)

  • Figure 1: A case of TinyLLaVA-Video-R1 on video understanding data, sourced from MVBench. The model demonstrates the ability to perceive video scenes and analyze options, while also exhibiting reflective and backtracking behavior (highlighted in blue).
  • Figure 2: A case of TinyLLaVA-Video-R1 on video reasoning data, sourced from MMVU. The model demonstrates comprehensive video content understanding and the capability to derive correct answers through analytical reasoning.
  • Figure 3: Cases of "aha moment", where the model demonstrates reflection and backtracking during its reasoning process (highlighted in blue). The cases are from MVBench and MMVU respectively.
  • Figure 4: Evolution in key metrics during the training of TinyLLaVA-Video-R1. Under our reward rule settings, both the response length and rewards of TinyLLaVA-Video-R1 gradually increased during training.
  • Figure 5: The variation in response length during training under different settings.
  • ...and 1 more figures