Reinforced Visual Perception with Tools

TL;DR

ReVPT introduces a reinforcement learning framework to enable multimodal language models to reason with external visual tools, addressing limitations of supervised fine-tuning. The approach combines a cold-start phase with GPT-4.1-synthesized tool trajectories and GRPO-based RL to learn adaptive tool use across four visual tools (object detection, zoom-in, edge detection, depth estimation). Empirical results show state-of-the-art performance on perception-centric benchmarks (e.g., CV-Bench, Blink-Hard) and notable improvements over instruct-tuned and text-based RL baselines, with 3B and 7B variants outperforming strong baselines on multiple tasks. The work emphasizes data design, tool selection, and the balance between general capability and specialized perception skills, and provides fully open-source code, datasets, and platform for community use.

Abstract

Visual reasoning, a cornerstone of human intelligence, encompasses complex perceptual and logical processes essential for solving diverse visual problems. While advances in computer vision have produced powerful models for various perceptual tasks, leveraging these for general visual reasoning remains challenging. Prior work demonstrates that augmenting LLMs with vision models via supervised finetuning improves performance, but faces key limitations such as expensive data generation, reliance on careful data filtering, and poor generalization. To address these issues, we propose ReVPT to enhance multi-modal LLMs' abilities to reason about and use visual tools through reinforcement learning. We introduce a novel RL algorithm based on GRPO, designed to train models to reason with a suite of four visual tools. Through extensive experiments, we show that our method achieves state-of-the-art performance on several perception-heavy benchmarks, including SAT, CV-Bench, BLINK and MMStar, significantly outperforming the supervised and text-based RL finetuning baselines. Notably, Our ReVPT-3B and ReVPT-7B outperform the instruct models by 9.03% and 9.44% on CV-Bench. Finally, we bring to the community new insights on RL-based visual tool-usage through extensive ablations. Our code is available at https://github.com/ls-kelvin/REVPT.

Figures (15)

• Figure 1: Our ReVPT-3B and 7B models outperform their instruct and text-only GRPO counterparts on perception-centric tasks while maintaining strong general capabilities across multiple benchmarks.
• Figure 2: ReVPT-3B and 7B are able to solve problems through thinking and selectively employing visual tools to reach strong visual perception. We illustrate some step-by-step visual reasoning breakdowns for challenging examples that model leveraging visual tools to solve complex perception tasks and outperform commercial models GPT-4.1.
• Figure 3: An overall pipeline of our ReVPT. (Top): Model-generated tool requests are managed by a local environment-based Tool Controller, which independently deploys visual tool services (e.g., Depth, Object Detection). These tools' outputs are then fed back to the LVLM for iterative reasoning. (Bottom): When processing a visual reasoning problem, ReVPT employs K-turn rollouts where the model interacts with the tool environment to learn an adaptive policy, culminating in the final model.
• Figure 4: Reinforced visual tool-usage training requires high-quality and verified data. We transform two datasets into multiple-choice question, then filter out too easy or hard questions for Qwen2.5-VL-7B by multiple inference.
• Figure 5: ReVPT-3B effectively learns tool utilization after cold-start process. ReVPT-3B shows bias toward object detection and depth estimation over zoom and edge detection tools due to cold-start data construction. Future work will address data balance and generalized perception objectives.