GRIT: Teaching MLLMs to Think with Images

TL;DR

This work tackles the lack of visually grounded reasoning in multimodal large language models by introducing GRIT, a framework that learns to think with images through reasoning traces that interleave natural language with explicit bounding-box coordinates. It pairs GRIT with GRPO-GR, a reinforcement learning algorithm that optimizes for final-answer accuracy and the grounded-format of the reasoning chain, avoiding the need for dense intermediate annotations. Empirical results on multiple VQA and grounding benchmarks show GRIT enables MLLMs to unify their grounding and reasoning capabilities with remarkable data efficiency—achieving strong performance using as few as 20 training triplets. The approach also reveals insights into how bounding boxes influence subsequent reasoning and how performance scales with data, highlighting both the potential and the limits of data-driven grounding in multimodal reasoning.

Abstract

Recent studies have demonstrated the efficacy of using Reinforcement Learning (RL) in building reasoning models that articulate chains of thoughts prior to producing final answers. However, despite ongoing advances that aim at enabling reasoning for vision-language tasks, existing open-source visual reasoning models typically generate reasoning content with pure natural language, lacking explicit integration of visual information. This limits their ability to produce clearly articulated and visually grounded reasoning chains. To this end, we propose Grounded Reasoning with Images and Texts (GRIT), a novel method for training MLLMs to think with images. GRIT introduces a grounded reasoning paradigm, in which models generate reasoning chains that interleave natural language and explicit bounding box coordinates. These coordinates point to regions of the input image that the model consults during its reasoning process. Additionally, GRIT is equipped with a reinforcement learning approach, GRPO-GR, built upon the GRPO algorithm. GRPO-GR employs robust rewards focused on the final answer accuracy and format of the grounded reasoning output, which eliminates the need for data with reasoning chain annotations or explicit bounding box labels. As a result, GRIT achieves exceptional data efficiency, requiring as few as 20 image-question-answer triplets from existing datasets. Comprehensive evaluations demonstrate that GRIT effectively trains MLLMs to produce coherent and visually grounded reasoning chains, showing a successful unification of reasoning and grounding abilities.

Figures (10)

• Figure 1: Comparison of reasoning with pure natural language and grounded reasoning from GRIT that mixes explicit bounding boxes for image regions with a chain of natural language thoughts. Our GRIT method enables MLLMs to perform grounded reasoning with only 20 training samples, realizing a clear and reliable process of thinking with images.
• Figure 2: Model update via GRPO-GR. During GRPO-GR training, we sample a group of model completions and calculate the grounded-reasoning-format reward ($r_{\text{format}}$), the optional grounded-target-counting reward ($r_{\text{count}}$), and the GPT-aided answer-accuracy reward ($r_{\text{ans}}$). The rewards are used to calculate the group-normalized advantage and guide the policy optimization.
• Figure 3: Inference examples of Qwen2.5-VL-GRIT.
• Figure 4: Correlation between image regions and "thoughts" in grounded reasoning evaluated by our Vision-Language Reasoning Cross-Modal Correlation metric. The result shows that models trained with GRIT outperform baselines.
• Figure 5: Model's average attention for image tokens during the generation of rethink segments. The overall higher curve for the original rethink segments of the GRIT-trained model shows that the bounding boxes generated facilitate stronger attention to the image input in subsequent reasoning.