UR-Bench: A Benchmark for Multi-Hop Reasoning over Ultra-High-Resolution Images

TL;DR

The paper tackles the problem of evaluating and advancing multimodal reasoning on ultra-high-resolution imagery, where existing benchmarks and end-to-end models struggle to preserve fine-grained visual information. It introduces UR-Bench, a benchmark with two categories, four subsets, and three reasoning levels to test multi-hop reasoning at gigapixel scales, complemented by an automated data engine for QA generation. The core contribution is an agent-based framework that uses Semantic Abstraction and Retrieval to map visual content into language space and orchestrates external Visual Tools for perception and reasoning. Experimental results show end-to-end MLLMs lag behind, while the proposed agent framework delivers substantial gains across subsets and reasoning levels, establishing a new state-of-the-art for ultra-high-resolution multimodal reasoning and highlighting the value of tool-augmented, high-resolution understanding.

Abstract

Recent multimodal large language models (MLLMs) show strong capabilities in visual-language reasoning, yet their performance on ultra-high-resolution imagery remains largely unexplored. Existing visual question answering (VQA) benchmarks typically rely on medium-resolution data, offering limited visual complexity. To bridge this gap, we introduce Ultra-high-resolution Reasoning Benchmark (UR-Bench), a benchmark designed to evaluate the reasoning capabilities of MLLMs under extreme visual information. UR-Bench comprises two major categories, Humanistic Scenes and Natural Scenes, covering four subsets of ultra-high-resolution images with distinct spatial structures and data sources. Each subset contains images ranging from hundreds of megapixels to gigapixels, accompanied by questions organized into three levels, enabling evaluation of models' reasoning capabilities in ultra-high-resolution scenarios. We further propose an agent-based framework in which a language model performs reasoning by invoking external visual tools. In addition, we introduce Semantic Abstraction and Retrieval tools that enable more efficient processing of ultra-high-resolution images. We evaluate state-of-the-art models using both an end-to-end MLLMs and our agent-based framework, demonstrating the effectiveness of our framework.

Figures (5)

• Figure 1: (a) Mean resolution (MP), file size (MB), and object count of high-resolution image benchmarks. (b) Two categories and four subsets of UR-Bench, with tasks organized across three difficulty levels.
• Figure 2: Overview of the data construction pipeline.
• Figure 3: Illustration of the agent-based framework for ultra-high-resolution image QA. The agent operates through natural-language reasoning and dynamically invokes external visual tools to handle large-scale images.
• Figure 4: Comparison of tool invocation frequencies under the Street-view Images subset, using different models as the decision model within our agent framework.
• Figure 5: A case from our agent framework under the Humanistic Scenes-Portrait Scrolls subset.