Rank-based Geographical Regularization: Revisiting Contrastive Self-Supervised Learning for Multispectral Remote Sensing Imagery

TL;DR

The paper tackles the challenge of learning robust representations from unlabeled multispectral RS imagery by injecting geographic structure into contrastive SSL. It introduces GeoRank, a rank-based regularizer that directly optimizes spherical distances between locations, aligning representation-space proximity with geodesic proximity through a locality-aware objective defined by $L_{GeoRank} = α L_{SSL} + (1 - α) L_{RankReg}$. GeoRank consistently improves a range of SSL frameworks across multiple downstream RS tasks, and the work further provides a systematic study of data augmentation, dataset cardinality, temporal views, and input size to offer practical guidelines for RS SSL. The findings highlight substantial efficiency gains and show that geographic priors can enhance performance when training and evaluation regions overlap, contributing to more effective and scalable remote sensing analysis.

Abstract

Self-supervised learning (SSL) has become a powerful paradigm for learning from large, unlabeled datasets, particularly in computer vision (CV). However, applying SSL to multispectral remote sensing (RS) images presents unique challenges and opportunities due to the geographical and temporal variability of the data. In this paper, we introduce GeoRank, a novel regularization method for contrastive SSL that improves upon prior techniques by directly optimizing spherical distances to embed geographical relationships into the learned feature space. GeoRank outperforms or matches prior methods that integrate geographical metadata and consistently improves diverse contrastive SSL algorithms (e.g., BYOL, DINO). Beyond this, we present a systematic investigation of key adaptations of contrastive SSL for multispectral RS images, including the effectiveness of data augmentations, the impact of dataset cardinality and image size on performance, and the task dependency of temporal views. Code is available at https://github.com/tomburgert/georank.

Figures (8)

• Figure 1: Overview of the proposed plug-in regularization term. Left: A contrastive SSL framework applying contrastive loss $L_{\text{SSL}}$ to the representation space. Right: The proposed regularization loss term $L_{\text{RankReg}}$ for an image $x_i$ in $B$. The order (rank) of distances on the Earth's surface space $\text{R}^\text{d}_i$ (measured by Haversine distance $d$) is used as a label for the order (rank) of distances in representation space $\text{R}^\text{s}_i$. The hyperparameter $m_i$ enables the loss when the distance of two locations on Earth is within the radius $d_{\text{max}}$. For simplicity, we denote the full vector $\mathbf{m}_i$ instead of individual entries $m_{ij}$.
• Figure 2: Performance comparison between the standard augmentation pipeline (blue), the geometric augmentation pipeline (green) and supervised training (red) on six classification downstream tasks when pre-training on SSL4EO: (a) evaluated with the evaluation protocol, (b) evaluated with the linear evaluation protocol, (c) evaluated with the fine-tuning protocol.
• Figure 3: t-SNE of penultimate layer representations for 2560 samples of BEN-V2 after PCA (50 components). Points are colored by normalized latitude in (a) GeoRank and (b) Baseline (MoCoV2).
• Figure 4: Performance of different subset sizes of the pre-training dataset (a) BEN-V2 and (b) SSL4EO, evaluated on six classification downstream tasks by .
• Figure 5: Example Sentinel-2 images taken from BEN-V2.