Toward task-driven satellite image super-resolution

TL;DR

The paper addresses the challenge of evaluating and training satellite image super-resolution (SR) when cross-sensor differences complicate pixel-wise ground-truth comparisons. It introduces a task-driven SR framework that uses a three-input evaluation setup ($ ext{LR}$, $S\\times$-upsampled, and $ ext{HR}$) across four downstream CV tasks, employing pre-trained models and adapting them to SR scales without new annotations. Key contributions include a scalable, annotation-free evaluation methodology, evidence that scale- and spectral-difference robustification via batch-normalization adaptation enables reuse of existing CV models, and empirical insights into how roads/buildings segmentation and keypoint detection respond to SR outputs. The work lays a roadmap for integrating task-driven losses and feature-space loss components with conventional SR losses, aiming to align SR outputs with practical downstream analysis in remote sensing. Overall, the approach helps bridge the gap between SR reconstruction quality and actionable, real-world image analysis in heterogeneous satellite data environments.

Abstract

Super-resolution is aimed at reconstructing high-resolution images from low-resolution observations. State-of-the-art approaches underpinned with deep learning allow for obtaining outstanding results, generating images of high perceptual quality. However, it often remains unclear whether the reconstructed details are close to the actual ground-truth information and whether they constitute a more valuable source for image analysis algorithms. In the reported work, we address the latter problem, and we present our efforts toward learning super-resolution algorithms in a task-driven way to make them suitable for generating high-resolution images that can be exploited for automated image analysis. In the reported initial research, we propose a methodological approach for assessing the existing models that perform computer vision tasks in terms of whether they can be used for evaluating super-resolution reconstruction algorithms, as well as training them in a task-driven way. We support our analysis with experimental study and we expect it to establish a solid foundation for selecting appropriate computer vision tasks that will advance the capabilities of real-world super-resolution.

Figures (5)

• Figure 1: A general outline of the evaluation procedure. The image analysis is performed from the LR image (purple line), its bicubically-upsampled version (green line), and from the HR image (blue line). The obtained outcomes are assessed in a subjective manner, and the expected quality ranking is as shown on the right. The blocks with rounded corners indicate the actions, while the remaining ones---the generated artifacts.
• Figure 2: Road segmentation performed for S-2 images (B08 band) from MuS2: original and bicubically-upsampled LR images, and the corresponding HR references, relying on different approaches to adapting the batch normalization parameters.
• Figure 3: Building segmentation performed for S-2 images (B08 band) from the MuS2 benchmark: original and bicubically-upsampled LR images, and the corresponding HR references, relying on different approaches to adapting the batch normalization parameters.
• Figure 4: Keypoints retrieved with Key.Net for (a) original S-2 image, (b) bicubically-upsampled image (by a factor of $3\times$, and (c) WorldView-2 HR reference. The color images were composed from S-2 B02, B03 and B04 bands at 10 m GSD.
• Figure 5: Results of unsupervised segmentation obtained with SA for an S-2 image (a, b) and for the HR reference (c, d).