Look Less, Reason More: Rollout-Guided Adaptive Pixel-Space Reasoning

TL;DR

This work tackles the challenge of fine-grained visual reasoning in Vision-Language Models by introducing adaptive pixel-space reasoning. It combines operation-aware supervised fine-tuning with rollout-guided reinforcement learning to learn query-specific decisions about when to perform pixel-level operations, balancing accuracy and efficiency. A dual-rollout reward scheme—pixel-necessity rollouts and adaptive rollouts—drives the model to invoke zoom-ins only when beneficial, resulting in superior performance across five multimodal benchmarks and markedly reduced tool usage. The approach demonstrates practical impact by improving reasoning in high-resolution and infographic-style tasks while conserving computational resources, offering a scalable solution for precise multimodal reasoning.

Abstract

Vision-Language Models (VLMs) excel at many multimodal tasks, yet they frequently struggle with tasks requiring precise understanding and handling of fine-grained visual elements. This is mainly due to information loss during image encoding or insufficient attention to critical regions. Recent work has shown promise by incorporating pixel-level visual information into the reasoning process, enabling VLMs to access high-resolution visual details during their thought process. However, this pixel-level information is often overused, leading to inefficiency and distraction from irrelevant visual details. To address these challenges, we propose the first framework for adaptive pixel reasoning that dynamically determines necessary pixel-level operations based on the input query. Specifically, we first apply operation-aware supervised fine-tuning to establish baseline competence in textual reasoning and visual operations, then design a novel rollout-guided reinforcement learning framework relying on feedback of the model's own responses, which enables the VLM to determine when pixel operations should be invoked based on query difficulty. Experiments on extensive multimodal reasoning benchmarks show that our model achieves superior performance while significantly reducing unnecessary visual operations. Impressively, our model achieves 73.4\% accuracy on HR-Bench 4K while maintaining a tool usage ratio of only 20.1\%, improving accuracy and simultaneously reducing tool usage by 66.5\% compared to the previous methods.

Figures (9)

• Figure 1: Comparison of different reasoning strategies. The “Overuse” strategy unnecessarily incorporates pixel-level operations, leading to inefficiency and potential distraction. The “Neglect” strategy relies solely on pure textual CoT reasoning, failing to engage with critical fine-grained visual details. Our “Adaptive” strategy achieves a balance by intelligently deciding whether to perform pixel-level operations based on the specific query, optimizing both accuracy and efficiency.
• Figure 2: Overview of rollout-guided reinforcement learning. The framework generates rollouts under three prompting modes: forced tool use, prohibited tool use, and adaptive tool use, and these rollouts are rewarded by multiple reward functions. The adaptive tool-necessity alignment reward leverages comparisons between tool and no-tool rollouts to determine pixel tool necessity and guide the adaptive rollout, where the reward is determined by the model’s own adaptive reasoning and match of tool necessity. All rewards are aggregated to compute group advantage, which updates the policy to achieve efficient and adaptive visual reasoning.
• Figure 3: Ablation study on the effectiveness of pixel necessity estimation, showing benchmark accuracy (a) and tool usage ratio (b).
• Figure 4: Comparison between Pixel Reasoner and our method on multimodal reasoning tasks. Left: Archaeological site sign text recognition. Right: Cricket statistics comparison.
• Figure A1: Case of license plate recognition task comparison between Pixel Reasoner and our method.