Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward

TL;DR

The paper addresses the bottleneck of multimodal perception in RLVR-trained MLLMs by introducing Perception-R1, which adds a visual perception reward based on consistency between extracted visual annotations and model outputs, judged by a language model. This approach mitigates reward sparsity and enhances both perception and reasoning, achieving state-of-the-art results on multiple multimodal math and general benchmarks with only 1,442 training samples, significantly outperforming data-hungry baselines. Key findings include the necessity of the visual reward and repetition penalty, robustness across benchmarks, and evidence of reward-hacking risks when using certain reward models, highlighting the importance of carefully designed verifiable visual cues. The work demonstrates a data-efficient pathway to strengthening multimodal perception foundations, ultimately improving rigorous reasoning in MLLMs.

Abstract

Enhancing the multimodal reasoning capabilities of Multimodal Large Language Models (MLLMs) is a challenging task that has attracted increasing attention in the community. Recently, several studies have applied Reinforcement Learning with Verifiable Rewards (RLVR) to the multimodal domain in order to enhance the reasoning abilities of MLLMs. However, these works largely overlook the enhancement of multimodal perception capabilities in MLLMs, which serve as a core prerequisite and foundational component of complex multimodal reasoning. Through McNemar's test, we find that existing RLVR method fails to effectively enhance the multimodal perception capabilities of MLLMs, thereby limiting their further improvement in multimodal reasoning. To address this limitation, we propose Perception-R1, which introduces a novel visual perception reward that explicitly encourages MLLMs to perceive the visual content accurately, thereby can effectively incentivizing both their multimodal perception and reasoning capabilities. Specifically, we first collect textual visual annotations from the CoT trajectories of multimodal problems, which will serve as visual references for reward assignment. During RLVR training, we employ a judging LLM to assess the consistency between the visual annotations and the responses generated by MLLM, and assign the visual perception reward based on these consistency judgments. Extensive experiments on several multimodal reasoning benchmarks demonstrate the effectiveness of our Perception-R1, which achieves state-of-the-art performance on most benchmarks using only 1,442 training data.

Figures (8)

• Figure 1: A comparison of three MLLMs on a geometry problem. Both Qwen2.5-VL-7B-IT and its RLVR-trained variant make severe perception errors but manage to guess the answer, whereas our Perception-R1 first accurately describes the image and then solves the problem correctly.
• Figure 2: Overview of training pipeline of the proposed Perception-R1. In addition to the accuracy and format rewards, we introduce a novel visual perception reward that explicitly guides MLLMs toward improving their multimodal perception capabilities.
• Figure 3: (a). Average performance across all benchmarks with varying $\gamma$ values. (b). Comparison of performance across all benchmarks when using different judging LLMs. (c). Dynamics of visual perception reward during training when using different judging LLMs.
• Figure 4: The template of human evaluation for multimodal perception.
• Figure 5: Comparison of Accuracy and Visual Perception Rewards between Perception-R1 and the variant using Qwen2.5-VL-32B-IT as the Reward Model.