Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Learning Meets Satellite Images -- An Evaluation on Handcrafted and Learning-based Features for Multi-date Satellite Stereo Images

Shuang Song, Luca Morelli, Xinyi Wu, Rongjun Qin, Hessah Albanwan, Fabio Remondino

TL;DR

This work evaluates handcrafted versus learning-based feature matching for off-track, multi-date satellite stereo to enable DSM generation. Using 496 WorldView-3 stereo pairs from the DFC2019 dataset, the authors implement a processing pipeline with SIFT and seven learning-based matchers, refine correspondences with Least Squares Matching, perform RPC-based relative orientation, and generate DSMs for comparison against LiDAR ground truth. Findings indicate that learning-based matchers provide robustness under extreme appearance changes, while traditional SIFT can still achieve competitive photogrammetric accuracy; detector-free methods like DKM often yield high inlier ratios, and LSM refinement consistently improves DSM quality across methods. The study informs feature matcher selection for satellite DSM pipelines and demonstrates that handcrafted methods remain relevant for scalable, cost-effective 3D reconstruction from off-track multi-date imagery.

Abstract

A critical step in the digital surface models(DSM) generation is feature matching. Off-track (or multi-date) satellite stereo images, in particular, can challenge the performance of feature matching due to spectral distortions between images, long baseline, and wide intersection angles. Feature matching methods have evolved over the years from handcrafted methods (e.g., SIFT) to learning-based methods (e.g., SuperPoint and SuperGlue). In this paper, we compare the performance of different features, also known as feature extraction and matching methods, applied to satellite imagery. A wide range of stereo pairs(~500) covering two separate study sites are used. SIFT, as a widely used classic feature extraction and matching algorithm, is compared with seven deep-learning matching methods: SuperGlue, LightGlue, LoFTR, ASpanFormer, DKM, GIM-LightGlue, and GIM-DKM. Results demonstrate that traditional matching methods are still competitive in this age of deep learning, although for particular scenarios learning-based methods are very promising.

Deep Learning Meets Satellite Images -- An Evaluation on Handcrafted and Learning-based Features for Multi-date Satellite Stereo Images

TL;DR

This work evaluates handcrafted versus learning-based feature matching for off-track, multi-date satellite stereo to enable DSM generation. Using 496 WorldView-3 stereo pairs from the DFC2019 dataset, the authors implement a processing pipeline with SIFT and seven learning-based matchers, refine correspondences with Least Squares Matching, perform RPC-based relative orientation, and generate DSMs for comparison against LiDAR ground truth. Findings indicate that learning-based matchers provide robustness under extreme appearance changes, while traditional SIFT can still achieve competitive photogrammetric accuracy; detector-free methods like DKM often yield high inlier ratios, and LSM refinement consistently improves DSM quality across methods. The study informs feature matcher selection for satellite DSM pipelines and demonstrates that handcrafted methods remain relevant for scalable, cost-effective 3D reconstruction from off-track multi-date imagery.

Abstract

A critical step in the digital surface models(DSM) generation is feature matching. Off-track (or multi-date) satellite stereo images, in particular, can challenge the performance of feature matching due to spectral distortions between images, long baseline, and wide intersection angles. Feature matching methods have evolved over the years from handcrafted methods (e.g., SIFT) to learning-based methods (e.g., SuperPoint and SuperGlue). In this paper, we compare the performance of different features, also known as feature extraction and matching methods, applied to satellite imagery. A wide range of stereo pairs(~500) covering two separate study sites are used. SIFT, as a widely used classic feature extraction and matching algorithm, is compared with seven deep-learning matching methods: SuperGlue, LightGlue, LoFTR, ASpanFormer, DKM, GIM-LightGlue, and GIM-DKM. Results demonstrate that traditional matching methods are still competitive in this age of deep learning, although for particular scenarios learning-based methods are very promising.
Paper Structure (13 sections, 2 equations, 14 figures, 2 tables)

This paper contains 13 sections, 2 equations, 14 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. The Proposed Processing and Evaluation Framework
  5. Satellite Off-track Stereo Pairs - Data Preparation
  6. Pair Matching with Handcrafted and Learning-based Features and Matchers
  7. Evaluation Metrics
  8. Experiments and Evaluation
  9. Datasets
  10. Analysis with Relative Orientation
  11. Analysis with Dense Stereo Matching
  12. Analysis of the Effectiveness of LSM for Point Localization Refinement
  13. Conclusions

Figures (14)

  • Figure 1: The evaluation workflow
  • Figure 2: An example of illumination difference (JAX, FL)
  • Figure 3: An example of seasonal difference (OMA, NE)
  • Figure 4: Image collection time within the DFC2019 dataset
  • Figure 5: Imaging properties of DFC2019 dataset
  • ...and 9 more figures