GeoViS: Geospatially Rewarded Visual Search for Remote Sensing Visual Grounding

TL;DR

GeoViS tackles the challenge of grounding in large-scale remote sensing imagery where targets are small and geospatial relations are complex. It reframes grounding as a geospatially rewarded visual search powered by a unified VisualRAG model that provides rewards, action guidance, and conditional grounding, enabling hierarchical, stepwise reasoning. The method achieves state-of-the-art results across five remote sensing grounding benchmarks and demonstrates strong cross-dataset generalization, highlighting its robustness and transferability. The approach offers interpretable, reward-guided exploration that increases effective resolution and precision for tiny targets in geospatial contexts, with broad practical implications for remote sensing analysis.

Abstract

Recent advances in multimodal large language models(MLLMs) have led to remarkable progress in visual grounding, enabling fine-grained cross-modal alignment between textual queries and image regions. However, transferring such capabilities to remote sensing imagery remains challenging, as targets are often extremely small within kilometer-scale scenes, and queries typically involve intricate geospatial relations such as relative positions, spatial hierarchies, or contextual dependencies across distant objects. To address these challenges, we propose GeoViS, a Geospatially Rewarded Visual Search framework that reformulates remote sensing visual grounding as a progressive search-and-reasoning process. Rather than directly predicting the target location in a single step, GeoViS actively explores the global image through a tree-structured sequence of visual cues, integrating multimodal perception, spatial reasoning, and reward-guided exploration to refine geospatial hypotheses iteratively. This design enables the model to detect subtle small-scale targets while maintaining holistic scene awareness. Extensive experiments on five remote sensing grounding benchmarks demonstrate that GeoViS achieves precise geospatial understanding and consistently surpasses existing methods across key visual grounding metrics, highlighting its strong cross-domain generalization and interpretability.

Figures (4)

• Figure 1: Complex queries with multi-object relations and tiny targets make remote sensing grounding challenging. While existing one-step methods that resize or divide images often fail, GeoViS parses structured semantics and performs reward-guided subregion exploration to achieve accurate localization.
• Figure 2: Overview of GeoViS. Complex queries are structured into object, position, and relation cues. GeoViS first performs MCTS-based visual search to identify the most informative subregion, where each node represents a candidate region, and then conducts conditional grounding using the global image and the selected subregion. The VisualRAG model supports the entire pipeline by providing action guidance, reward evaluation, and final localization.
• Figure 3: Qualitative results on DIOR-RSVG, OPT-RSVG, and RSVG-HR comparing baseline (Qwen-2.5-VL-3B) with GeoViS.
• Figure 4: Ablation on the reward balance ratio $\alpha$ on DIOR-RSVG. The vertical axis shows the relative performance change (%) normalized to the maximum value for better visualization.