PixelThink: Towards Efficient Chain-of-Pixel Reasoning

TL;DR

PixelThink introduces an efficiency-aware reinforcement-finetuning scheme for reasoning segmentation, regulating chain-of-pixel reasoning through task difficulty and model uncertainty. It augments GRPO with a soft length penalty and a difficulty-aware reward to learn concise, task-adaptive reasoning that still supports accurate pixel-level masks. The authors also present ReasonSeg-Diff, a difficulty-annotated benchmark with short/long reference reasoning for holistic evaluation of segmentation quality, reasoning, and efficiency. Experiments show reduced reasoning tokens and improved segmentation accuracy across easy, medium, and hard cases, with strong generalization to standard RE-Seg benchmarks. This work advances interpretable, token-efficient multimodal grounding in vision-language tasks.

Abstract

Existing reasoning segmentation approaches typically fine-tune multimodal large language models (MLLMs) using image-text pairs and corresponding mask labels. However, they exhibit limited generalization to out-of-distribution scenarios without an explicit reasoning process. Although recent efforts leverage reinforcement learning through group-relative policy optimization (GRPO) to enhance reasoning ability, they often suffer from overthinking - producing uniformly verbose reasoning chains irrespective of task complexity. This results in elevated computational costs and limited control over reasoning quality. To address this problem, we propose PixelThink, a simple yet effective scheme that integrates externally estimated task difficulty and internally measured model uncertainty to regulate reasoning generation within a reinforcement learning paradigm. The model learns to compress reasoning length in accordance with scene complexity and predictive confidence. To support comprehensive evaluation, we introduce ReasonSeg-Diff, an extended benchmark with annotated reasoning references and difficulty scores, along with a suite of metrics designed to assess segmentation accuracy, reasoning quality, and efficiency jointly. Experimental results demonstrate that the proposed approach improves both reasoning efficiency and overall segmentation performance. Our work contributes novel perspectives towards efficient and interpretable multimodal understanding. The code and model will be publicly available.

Figures (7)

• Figure 1: Motivation of Efficient Chain-of-Pixel Reasoning (PixelThink). We propose a novel scheme for reasoning segmentation that effectively regulates reasoning length based on task difficulty and uncertainty. Our method improves segmentation quality while significantly reducing token usage. A suite of metrics is introduced for holistic evaluations of reasoning quality, segmentation accuracy, and computational efficiency.
• Figure 2: Overview of PixelThink. (a) Workflow of the reasoning segmentation framework. Given an input image and query, the model generates a reasoning chain and segmentation reference that guides the segmentation outcome. (b) The group-relative policy optimization (GRPO) procedure employed during reinforcement fine-tuning. Our new reward design incorporates both task difficulty and model uncertainty, enabling the model to learn efficient reasoning strategies.
• Figure 3: The construction of ReasonSeg-Diff. (a) Design on the Difficulty Scoring scheme. (b) Generation of the Short Thinking process. (c) Generation of Long Thinking process.
• Figure 4: Qualitative comparisons between Seg-Zero liu2025seg and the proposed PixelThink. Representative samples across different difficulty levels are selected to highlight differences in the reasoning process and segmentation performance.
• Figure A: Qualitative examples from ReasonSeg-Diff. Representative samples are shown with annotated difficulty scores and corresponding reference reasoning chains. For easy and medium cases, short reasoning chains are preferred to capture concise visual cues, whereas hard cases benefit from longer chains that reflect more elaborate reasoning over ambiguous or complex scenes. These examples illustrate the diverse reasoning requirements across difficulty levels and support more fine-grained evaluation.