GeoGround: A Unified Large Vision-Language Model for Remote Sensing Visual Grounding

TL;DR

GeoGround presents a unified large vision-language model for remote sensing visual grounding by textually encoding HBB, OBB, and mask signals into a single training pipeline. The Text-Mask paradigm enables pixel-level outputs without extra decoders, while PAL and GGL promote cross-signal consistency, reinforced by a BBox Consistency Score. A large-scale refGeo dataset supports diverse RS grounding tasks, and experiments show GeoGround achieving state-of-the-art or competitive performance across REC, OBB, and RES benchmarks, with strong generalization to small objects. This work advances RS grounding by enabling multi-type outputs within a single, efficient VLM framework and providing a valuable curriculum for future RS VLM research.

Abstract

Remote sensing (RS) visual grounding aims to use natural language expression to locate specific objects (in the form of the bounding box or segmentation mask) in RS images, enhancing human interaction with intelligent RS interpretation systems. Early research in this area was primarily based on horizontal bounding boxes (HBBs), but as more diverse RS datasets have become available, tasks involving oriented bounding boxes (OBBs) and segmentation masks have emerged. In practical applications, different targets require different grounding types: HBB can localize an object's position, OBB provides its orientation, and mask depicts its shape. However, existing specialized methods are typically tailored to a single type of RS visual grounding task and are hard to generalize across tasks. In contrast, large vision-language models (VLMs) exhibit powerful multi-task learning capabilities but struggle to handle dense prediction tasks like segmentation. This paper proposes GeoGround, a novel framework that unifies support for HBB, OBB, and mask RS visual grounding tasks, allowing flexible output selection. Rather than customizing the architecture of VLM, our work aims to elegantly support pixel-level visual grounding output through the Text-Mask technique. We define prompt-assisted and geometry-guided learning to enhance consistency across different signals. Experimental results show that GeoGround demonstrates strong performance across four RS visual grounding tasks, matching the performance of specialized methods on multiple benchmarks. Code available at https://github.com/zytx121/GeoGround

Figures (5)

• Figure 1: An overview of the hybrid supervision of GeoGround.
• Figure 2: An overview of GeoGround -- the first model to unify box-level and pixel-level visual grounding tasks in remote sensing.
• Figure 3: Illustrations of PAL and GGL. The concepts of sparse and dense signals are relative. For example, the sparse signal corresponds to HBB, while the dense signal corresponds to OBB.
• Figure 4: Visualizations of GeoGround and GeoChat on the REC task with HBB and OBB.
• Figure 5: Visualizations of GeoGround with different resolutions of Text-Mask on the RRSIS-D test set.