Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodology

TL;DR

This work introduces TreeBench, the first benchmark explicitly designed to evaluate visual grounded reasoning with traceable evidence, emphasizing small-target perception, multi-step reasoning, and bounding-box explainability. It pairs TreeBench with TreeVGR, a two-stage training framework that first cold-starts with supervised grounding and then optimizes reasoning trajectories via reinforcement learning using a dual IoU reward to ensure precise and traceable localization. The approach yields significant gains across high-resolution benchmarks and demonstrates a positive link between localization accuracy and reasoning performance, establishing a path toward more transparent and capable vision-language systems. The work also discusses limitations (7B-scale models, 405 samples) and points to future work to expand benchmarks and scalability.

Abstract

Models like OpenAI-o3 pioneer visual grounded reasoning by dynamically referencing visual regions, just like human "thinking with images". However, no benchmark exists to evaluate these capabilities holistically. To bridge this gap, we propose TreeBench (Traceable Evidence Evaluation Benchmark), a diagnostic benchmark built on three principles: (1) focused visual perception of subtle targets in complex scenes, (2) traceable evidence via bounding box evaluation, and (3) second-order reasoning to test object interactions and spatial hierarchies beyond simple object localization. Prioritizing images with dense objects, we initially sample 1K high-quality images from SA-1B, and incorporate eight LMM experts to manually annotate questions, candidate options, and answers for each image. After three stages of quality control, TreeBench consists of 405 challenging visual question-answering pairs, even the most advanced models struggle with this benchmark, where none of them reach 60% accuracy, e.g., OpenAI-o3 scores only 54.87. Furthermore, we introduce TreeVGR (Traceable Evidence Enhanced Visual Grounded Reasoning), a training paradigm to supervise localization and reasoning jointly with reinforcement learning, enabling accurate localizations and explainable reasoning pathways. Initialized from Qwen2.5-VL-7B, it improves V* Bench (+16.8), MME-RealWorld (+12.6), and TreeBench (+13.4), proving traceability is key to advancing vision-grounded reasoning. The code is available at https://github.com/Haochen-Wang409/TreeVGR.

Figures (24)

• Figure 1: Qualitative examples from TreeBench for each discipline. Each question requires focused visual parsing on mere objects, and some even request second-order reasoning beyond precise localization. Moreover, the bounding boxes of all target instances are provided, ensuring a traceable evaluation. All these questions are challenging, as OpenAI-o3 o3 and Gemini-2.5-Pro gemini-2.5-procannot answer them correctly simultaneously.
• Figure 2: Normalized performance comparison with our TreeVGR and other works bai2025qwen25vlzheng2025deepeyessu2025pixelreasoner on our TreeBench for each category.
• Figure 3: Distribution of each discipline in TreeBench, which prioritizes reasoning over perception.
• Figure 4: The ground-truth distribution of TreeBench with 3 instances of E and 4 instances of F.
• Figure 5: Distribution of the number of instances in TreeBench, with one question with 8 target instances.