SatDepth: A Novel Dataset for Satellite Image Matching

TL;DR

SatDepth introduces a novel dense ground-truth satellite image matching dataset built from stereo-derived DSMs, enabling pixel-level correspondences across multi-view satellite imagery. The authors develop a processing pipeline (tiling, BA alignment, stereo matching, DSM fusion) and a SatDepth Map representation (lat-lon-height per pixel), plus a rotation-augmentation technique to address track-angle imbalances. They benchmark four state-of-the-art matchers trained from scratch on SatDepth and demonstrate that rotation augmentation markedly improves performance, including under large rotational differences and unseen AOIs. The dataset is validated with LiDAR-based DSM comparisons and extensive GCP annotations, achieving sub-meter relative accuracy and enabling broader generalization for satellite image matching tasks. This work provides a practical, scalable resource and protocol for advancing learned image matching in satellite imagery, with implications for geospatial alignment and 3D reconstruction workflows.

Abstract

Recent advances in deep-learning based methods for image matching have demonstrated their superiority over traditional algorithms, enabling correspondence estimation in challenging scenes with significant differences in viewing angles, illumination and weather conditions. However, the existing datasets, learning frameworks, and evaluation metrics for the deep-learning based methods are limited to ground-based images recorded with pinhole cameras and have not been explored for satellite images. In this paper, we present ``SatDepth'', a novel dataset that provides dense ground-truth correspondences for training image matching frameworks meant specifically for satellite images. Satellites capture images from various viewing angles and tracks through multiple revisits over a region. To manage this variability, we propose a dataset balancing strategy through a novel image rotation augmentation procedure. This procedure allows for the discovery of corresponding pixels even in the presence of large rotational differences between the images. We benchmark four existing image matching frameworks using our dataset and carry out an ablation study that confirms that the models trained with our dataset with rotation augmentation outperform (up to 40% increase in precision) the models trained with other datasets, especially when there exist large rotational differences between the images.

Figures (43)

• Figure 1: Image matches learned by satLoFTR trained using SatDepth dataset with rotation augmentation for image pairs with significant differences. The green lines depict only 40 randomly chosen correctly detected matches.
• Figure 2: Spatial extents of each AOI (red box) in the SatDepth dataset overlaid on Bing Maps.
• Figure 3: SatDepth processing pipeline: given a collection of satellite images (with cameras (RPCs) and metadata (IMDs)) and auxiliary data (DEM and Water Mask), we carry out a series of processing steps to obtain SatDepth Maps which are used to extract ground-truth correspondences.
• Figure 4: Comparison of the quality of the DSM generated from different stereo pipelines over San Fernando.
• Figure 5: GCP error summary for Jacksonville AOI. A total of 76 GCPs were annotated on satellite images to measure accuracy.