Fast-Slow Thinking GRPO for Large Vision-Language Model Reasoning

TL;DR

FAST proposes a fast-slow thinking framework for large vision-language models by dynamically adjusting reasoning depth according to question difficulty. It introduces two complementary metrics to estimate multimodal question difficulty and integrates adaptive length-based rewards with a difficulty-aware KL term into GRPO, achieving state-of-the-art accuracy while substantially reducing output length. Across seven benchmarks, FAST improves average accuracy by over 10% with significant token savings, and scales to 32B-parameter models with favorable accuracy-length trade-offs. The approach highlights the importance of data distribution, intrinsic and extrinsic difficulty, and input-grounding quality for robust multimodal reasoning.

Abstract

When applying reinforcement learning--typically through GRPO--to large vision-language model reasoning struggles to effectively scale reasoning length or generates verbose outputs across all tasks with only marginal gains in accuracy. To address this issue, we present FAST-GRPO, a variant of GRPO that dynamically adapts reasoning depth based on question characteristics. Through empirical analysis, we establish the feasibility of fast-slow thinking in LVLMs by investigating how response length and data distribution affect performance. Inspired by these observations, we introduce two complementary metrics to estimate the difficulty of the questions, guiding the model to determine when fast or slow thinking is more appropriate. Next, we incorporate adaptive length-based rewards and difficulty-aware KL divergence into the GRPO algorithm. Experiments across seven reasoning benchmarks demonstrate that FAST achieves state-of-the-art accuracy with over 10\% relative improvement compared to the base model, while reducing token usage by 32.7-67.3\% compared to previous slow-thinking approaches, effectively balancing reasoning length and accuracy.

Figures (15)

• Figure 1: FAST achieves higher average accuracy with shorter average response lengths across seven benchmarks. All methods are built upon Qwen2.5-VL.
• Figure 2: Effect of length rewards on reasoning length and accuracy.
• Figure 3: Effect of data distribution, especially difficulty on reasoning length and accuracy.
• Figure 4: Left: Results on the effect of difficulty threshold. The average accuracy is computed across MathVision, MathVerse, and MathVista. Middle: Test set results with different difficulty level training split comparisons on Geometry 3K. Right: Test set results with different $\beta$ value comparison in pilot experiments on Geometry 3K. OOD results comparison on MM-Vet benchmark.
• Figure 5: Error breakdown by category.