EarthScape: A Multimodal Dataset for Surficial Geologic Mapping and Earth Surface Analysis

TL;DR

EarthScape introduces a first AI-ready, multimodal dataset tailored for surficial geologic mapping and Earth surface analysis by fusing high-resolution RGB and NIR imagery, DEMs, terrain features across multiple scales, and GIS vector data for hydrology and infrastructure. The dataset comprises 31,018 patches across Warren and Hardin counties, annotated with seven geologic classes and 38 input channels, enabling multilabel classification and potential segmentation tasks. Baseline experiments with SGMap-Net show that DEM and elevation-derived features provide strong in-domain performance, while cross-domain generalization remains challenging and multimodal Early Fusion can underperform in unseen regions, underscoring the need for domain-robust fusion strategies. EarthScape is designed as a living benchmark to spur multimodal learning, domain adaptation, and high-resolution geospatial analysis in geosciences with broad potential for future expansion and pretrained regional models.

Abstract

Surficial geologic mapping is essential for understanding Earth surface processes, addressing modern challenges such as climate change and national security, and supporting common applications in engineering and resource management. However, traditional mapping methods are labor-intensive, limiting spatial coverage and introducing potential biases. To address these limitations, we introduce EarthScape, a novel, AI-ready multimodal dataset specifically designed for surficial geologic mapping and Earth surface analysis. EarthScape integrates high-resolution aerial RGB and near-infrared (NIR) imagery, digital elevation models (DEM), multi-scale DEM-derived terrain features, and hydrologic and infrastructure vector data. The dataset provides detailed annotations for seven distinct surficial geologic classes encompassing various geological processes. We present a comprehensive data processing pipeline using open-sourced raw data and establish baseline benchmarks using different spatial modalities to demonstrate the utility of EarthScape. As a living dataset with a vision for expansion, EarthScape bridges the gap between computer vision and Earth sciences, offering a valuable resource for advancing research in multimodal learning, geospatial analysis, and geological mapping. Our code is available at https://github.com/masseygeo/earthscape.

Figures (12)

• Figure 1: Surficial geologic map showing the seven target classes. The mask is displayed with transparency over hillshade to provide a visual reference between geology and landscape. The grid represents selected EarthScape patches, each measuring 1280 feet (256 pixels) with 50% overlap; red square in upper left shows an example of one patch.
• Figure 2: Target mask and selected modalities for one randomly selected EarthScape patch. (top) From left to right: target mask (same colors as Fig. \ref{['fig: geo_map']}, RGB aerial imagery, DEM, and slope. (bottom) From left to right: Profile curvatures from the 20-foot and 100-foot DEMs, and elevation percentile calculated with 51$\times$51 and 201$\times$201 windows.
• Figure 3: Data processing pipeline for EarthScape.
• Figure 4: Dataset statistics of total class counts (upper left), distributions of class proportions per patch (bottom left), and number of classes per patch (right). Colors correspond to Figure \ref{['fig: geo_map']}.
• Figure 5: Surficial geologic map of part of Hardin County showing six target classes. As seen in the main paper, this map highlights similar terrain characteristics and surficial geologic units. The geologic mask is overlaid with transparency on a hillshade image to highlight the relationship between geologic features and landscape. The grid represents EarthScape patches, each measuring 1280 feet (256 pixels) with 50% overlap. The red square in the upper left outlines one example patch for scale.