SATORI-R1: Incentivizing Multimodal Reasoning through Explicit Visual Anchoring

TL;DR

SATORI introduces a spatially grounded, reinforcement-learning framework for multimodal reasoning by replacing free-form RL reasoning with a Glance–Focus–Think process. It leverages verifiable intermediate signals—image captions and bounding boxes—as dense rewards, reducing gradient variance and improving attention to task-relevant regions. The authors provide the VQA-Verify dataset to support training with explicit grounding supervision and demonstrate consistent gains across seven multimodal benchmarks, including notable improvements in visual and mathematical reasoning. The work highlights causal evidence that enforced visual grounding drives accuracy and offers robust ablations on reward design and reasoning sequences, suggesting practical viability for scalable, grounded multimodal reasoning.

Abstract

DeepSeek-R1 has demonstrated powerful reasoning capabilities in the text domain through stable reinforcement learning (RL). Recently, in the multimodal domain, works have begun to directly apply RL to generate R1-like free-form reasoning for Visual Question Answering (VQA) tasks. However, multimodal tasks share an intrinsically different nature from textual tasks, which heavily rely on the understanding of the input image to solve the problem. Therefore, such free-form reasoning faces two critical limitations in the VQA task: (1) Extended reasoning chains diffuse visual focus away from task-critical regions, degrading answer accuracy. (2) Unverifiable intermediate steps amplify policy-gradient variance and computational costs overhead. To address these issues, in this paper, we introduce SATORI ($\textbf{S}patially$ $\textbf{A}nchored$ $\textbf{T}ask$ $\textbf{O}ptimization$ with $\textbf{R}e\textbf{I}nforcement$ Learning), which decomposes VQA into three verifiable stages, including global image captioning, region localization, and answer prediction, each supplying explicit reward signals. Furthermore, we also introduce VQA-Verify, a 12k dataset annotated with answer-aligned captions and bounding-boxes to facilitate training. Experiments demonstrate consistent performance improvements across seven VQA benchmarks, achieving up to $15.7\%$ improvement in accuracy in accuracy compared to the R1-like baseline. Our analysis of the attention map confirms enhanced focus on critical regions, which brings improvements in accuracy. Our code is available at https://github.com/justairr/SATORI-R1.

Figures (8)

• Figure 1: Comparison of Our Reasoning Patterns and Free-form Reasoning. Using the same model Qwen2.5-VL-Instruct-3B with only the output patterns altered, the Flow of Visual Focus heatmaps for free-form reasoning show that attention becomes progressively diffuse and scattered as the reasoning chain lengthens. In contrast, our Glance $\rightarrow$ Focus $\rightarrow$ Think paradigm guides the model's attention from a holistic view to a focused concentration on task-relevant regions. Each attention map is obtained by aggregating approximately 40 tokens output by the model.
• Figure 2: RAD and accuracy distributions for three different reasoning types. The light‑shaded region represents the 95% confidence interval.
• Figure 3: The overview of our proposed method. SATORI guides the model to capture the global information, then analyzes task-relevant regions and finally produces an answer, providing verifiable rewards for step-by-step supervision.
• Figure 4: Overview of VQA-Verify. VQA-Verify is divided into 3 categories, 11 subtasks, and 17 benchmarks in total.
• Figure 5: Performance of different methods across various reasoning and OCR benchmarks on Qwen-2.5-VL-Instruct-3B. Specifically, MME$^{R}$, MME$^{\text{Code}}$ and MME$^{CR}$ denote the Reasoning, Code Reasoning and Commonsense Reasoning of MME fu2024mmecomprehensiveevaluationbenchmark.