Extending global-local view alignment for self-supervised learning with remote sensing imagery

TL;DR

This work extends self-supervised learning for remote sensing by adding two global-local view strategies to DINO: temporal-positive contrast (DINO-TP) and multi-sized local crops (DINO-MC). DINO-MC, in particular, uses a variety of local crop sizes to better capture object-scale variation in RS imagery and applies distinct augmentations to global and local views, achieving strong transfer to land-use classification and change-detection tasks with limited pre-training data. Across EuroSAT, BigEarthNet, and OSCD, DINO-MC generally outperforms prior RS SSL methods and DINO variants, while DINO-TP provides insights but can be unstable for some tasks. The results suggest substantial practical impact for learning robust RS representations with reduced labeling and computational needs, with code released at the cited GitHub repository.

Abstract

Since large number of high-quality remote sensing images are readily accessible, exploiting the corpus of images with less manual annotation draws increasing attention. Self-supervised models acquire general feature representations by formulating a pretext task that generates pseudo-labels for massive unlabeled data to provide supervision for training. While prior studies have explored multiple self-supervised learning techniques in remote sensing domain, pretext tasks based on local-global view alignment remain underexplored, despite achieving state-of-the-art results on natural imagery. Inspired by DINO, which employs an effective representation learning structure with knowledge distillation based on global-local view alignment, we formulate two pretext tasks for self-supervised learning on remote sensing imagery (SSLRS). Using these tasks, we explore the effectiveness of positive temporal contrast as well as multi-sized views on SSLRS. We extend DINO and propose DINO-MC which uses local views of various sized crops instead of a single fixed size in order to alleviate the limited variation in object size observed in remote sensing imagery. Our experiments demonstrate that even when pre-trained on only 10% of the dataset, DINO-MC performs on par or better than existing state-of-the-art SSLRS methods on multiple remote sensing tasks, while using less computational resources. All codes, models, and results are released at https://github.com/WennyXY/DINO-MC.

Figures (3)

• Figure 1: DINO: the self-supervised contrastive algorithm with knowledge distillation. It is the basic structure of both DINO-TP and DINO-MC. For DINO-TP, we use three temporal views to generate global crops and multi-sizes local crops as the input to do positive contrastive representation learning. For DINO-MC, we generate global and local crops from one imagery, then apply two different augmentations to global views and multi-sizes local views, respectively, to get the input of teacher and student network.
• Figure 2: The process of handling temporal views of contrastive learning in DINO-TP. We randomly select three temporal views of the same location and augment them to obtain global and local crops, which is the input of the teacher and student network. Different temporal views of the same location in DINO-TP are considered as positive examples, and we train the model to match their representations in the feature space.
• Figure 3: Same as SeCo manas2021seasonal, our visualization is based on two instances of 'Losvegas' and 'Dubai' from the OSCD change detection dataset. SeCo-1M is SeCo model pre-trained on SeCo-1M dataset, while DINO, DINO-TP, and DINO-MC are pre-trained on only SeCo-100K. We visualize the outputs of DINO-TP and DINO-MC for comparison with DINO and SeCo. We present results on the same images as SeCo for comparison and the two outputs of SeCo is from manas2021seasonal. We also provide F1 score for each output on the bottom of output image.