OpenVLThinkerV2: A Generalist Multimodal Reasoning Model for Multi-domain Visual Tasks

Abstract

Group Relative Policy Optimization (GRPO) has emerged as the de facto Reinforcement Learning (RL) objective driving recent advancements in Multimodal Large Language Models. However, extending this success to open-source multimodal generalist models remains heavily constrained by two primary challenges: the extreme variance in reward topologies across diverse visual tasks, and the inherent difficulty of balancing fine-grained perception with multi-step reasoning capabilities. To address these issues, we introduce Gaussian GRPO (G$^2$RPO), a novel RL training objective that replaces standard linear scaling with non-linear distributional matching. By mathematically forcing the advantage distribution of any given task to strictly converge to a standard normal distribution, $\mathcal{N}(0,1)$, G$^2$RPO theoretically ensures inter-task gradient equity, mitigates vulnerabilities to heavy-tail outliers, and offers symmetric update for positive and negative rewards. Leveraging the enhanced training stability provided by G$^2$RPO, we introduce two task-level shaping mechanisms to seamlessly balance perception and reasoning. First, response length shaping dynamically elicits extended reasoning chains for complex queries while enforce direct outputs to bolster visual grounding. Second, entropy shaping tightly bounds the model's exploration zone, effectively preventing both entropy collapse and entropy explosion. Integrating these methodologies, we present OpenVLThinkerV2, a highly robust, general-purpose multimodal model. Extensive evaluations across 18 diverse benchmarks demonstrate its superior performance over strong open-source and leading proprietary frontier models.

Figures (8)

• Figure 1: Performance improvement (relative) of OpenVLThinkerV2 over its baseline Qwen3-VL-Instruct-8B across diverse visual tasks.
• Figure 2: Comparison of advantage formulations against previous methods. By enforcing a Gaussian topology, G$^2$RPO provides 1) intrinsic robustness to outliers, 2) symmetric updates for positive and negative rewards, and 3) uniform variance across diverse tasks.
• Figure 3: Comparison of response length dynamics during training. G$^2$RPO effectively encourage early convergence. a) It scales up reasoning length for complex question. b) It reduces overthinking for visual-centric tasks, enhancing perceptual grounding and mitigating hallucinations. c) For both reasoning and perception heavy tasks, the generation length stabilizes within an optimal range, effectively balancing both capabilities.
• Figure 4: Effect of task-level entropy shaping. G$^2$RPO effectively prevents entropy explosion for reasoning-centric tasks and OOD task (spatial reasoning) while concurrently mitigating entropy collapse in vision-centric tasks.
• Figure 5: Average Accuracy Reward Comparison across all tasks on the Validation set during Training. G$^2$RPO demonstrates stable and superior performance overall.