OpenEarthAgent: A Unified Framework for Tool-Augmented Geospatial Agents

TL;DR

OpenEarthAgent introduces a unified framework for developing tool-augmented geospatial agents trained on satellite imagery, natural-language queries, and detailed reasoning traces that demonstrates structured reasoning, stable spatial understanding, and interpretable behaviour through tool-driven geospatial interactions across diverse conditions.

Abstract

Recent progress in multimodal reasoning has enabled agents that can interpret imagery, connect it with language, and perform structured analytical tasks. Extending such capabilities to the remote sensing domain remains challenging, as models must reason over spatial scale, geographic structures, and multispectral indices while maintaining coherent multi-step logic. To bridge this gap, OpenEarthAgent introduces a unified framework for developing tool-augmented geospatial agents trained on satellite imagery, natural-language queries, and detailed reasoning traces. The training pipeline relies on supervised fine-tuning over structured reasoning trajectories, aligning the model with verified multistep tool interactions across diverse analytical contexts. The accompanying corpus comprises 14,538 training and 1,169 evaluation instances, with more than 100K reasoning steps in the training split and over 7K reasoning steps in the evaluation split. It spans urban, environmental, disaster, and infrastructure domains, and incorporates GIS-based operations alongside index analyses such as NDVI, NBR, and NDBI. Grounded in explicit reasoning traces, the learned agent demonstrates structured reasoning, stable spatial understanding, and interpretable behaviour through tool-driven geospatial interactions across diverse conditions. We report consistent improvements over a strong baseline and competitive performance relative to recent open and closed-source models.

Figures (12)

• Figure 1: Comparison between baseline Qwen3-4B and OpenEarthAgent-4B on a GIS reasoning query. OpenEarthAgent executes tools sequentially with correct dependencies and interpretable feedback, while Qwen3 fails due to unordered multi-tool calls and inconsistent reasoning.
• Figure 2: Unified data-curation pipeline for OpenEarthAgent Dataset. The process integrates diverse data sources: RGB-SAR imagery, GIS-based spatial layers, and index-driven datasets to build a harmonized geospatial corpus. Each branch performs its own filtering: RGB-SAR samples undergo sufficiency and annotation checks; GIS scenes are selected using point-of-interest (POI) constraints; and index-based inputs capture spectral changes (e.g., NDVI, NBR, NDBI). The outputs are merged into unified JSON records containing imagery metadata and geopackage information. A question synthesizer and conversational one-shot generator create tool-grounded queries and reasoning traces, which are validated through an automated suite ensuring geographic and syntactic correctness. Invalid instances are discarded, while verified ones form the final high-quality corpus used for agentic training and evaluation.
• Figure 3: Representative examples of query-reasoning trajectories from the proposed dataset. Each example illustrates the interleaving of natural-language thoughts, tool executions, and intermediate observations across diverse modalities and task types, including object localization, distance measurement, spectral index mapping, change detection, and GIS-based spatial analysis. These samples depict simplified excerpts, while the full dataset contains longer, more comprehensive multi-step reasoning traces.
• Figure 4: Key statistics for the curated corpus used for training and evaluation of our proposed agentic pipeline.
• Figure 5: Overview of the proposed OpenEarthAgent framework. The figure depicts tool deployment, where user queries over RGB, SAR, paired change-detection (CD), or GIS/indexed imagery are processed by the geospatial reasoning engine and tool orchestrator, which invoke appropriate tools and integrate feedback.