Transfer Learning for Onboard Cloud Segmentation in Thermal Earth Observation: From Landsat to a CubeSat Constellation

TL;DR

This work tackles onboard cloud segmentation for thermal Earth observation on CubeSats with limited hardware and a single thermal band. It employs transfer learning by pretraining a UNet with a MobileNet encoder on Landsat-7 Cloud Cover Assessment data and fine-tuning with a small FOREST-2 dataset in a joint-training setup. The results show a macro F1 increase from 0.850 to 0.877 and improved AP/AUC (0.92/0.96), confirming cross-domain benefit and robust performance with limited labeled data. Importantly, the model runs onboard in under 5 seconds on a Jetson Nano using tiling and TensorRT, enabling real-time cloud masking for data prioritization in bandwidth-constrained missions.

Abstract

Onboard cloud segmentation is a critical yet underexplored task in thermal Earth observation (EO), particularly for CubeSat missions constrained by limited hardware and spectral information. CubeSats often rely on a single thermal band and lack sufficient labeled data, making conventional cloud masking techniques infeasible. This work addresses these challenges by applying transfer learning to thermal cloud segmentation for the FOREST-2 CubeSat, using a UNet with a lightweight MobileNet encoder. We pretrain the model on the public Landsat-7 Cloud Cover Assessment Dataset and fine-tune it with a small set of mission-specific samples in a joint-training setup, improving the macro F1 from 0.850 to 0.877 over FOREST-2-only baselines. We convert the model to a TensorRT engine and demonstrate full-image inference in under 5 seconds on an NVIDIA Jetson Nano. These results show that leveraging public datasets and lightweight architectures can enable accurate, efficient thermal-only cloud masking on-orbit, supporting real-time decision-making in data-limited EO missions.

Figures (4)

• Figure 1: Macro F1 and accuracy across six folds for models trained on FOREST-2 only (f2-f2) and jointly with Landsat-7 (joint-f2). Joint training improves macro F1 from 0.850 to 0.877 and accuracy from 0.889 to 0.910. Dashed lines show Landsat-only results: l7-l7 (green) and l7-f2 (red).
• Figure 2: Onboard image tiling and stitching scheme. Red tiles are 512x512px, black tiles are 256x256px. The resulting cloud mask is shown in white.
• Figure 3: Qualitative comparison of cloud segmentation results on five representative FOREST-2 crops. Each row shows the original thermal input, the corresponding manual annotation, and the predicted masks from the evaluated models. The joint-trained model performs particularly well in visually complex regions and appears to handle thin cloud structures more accurately, likely due to the greater presence of such cases in the Landsat-7 training data. Notably, the l7-f2 model also produces reasonable segmentation results, despite having never seen FOREST-2 data during training.
• Figure 4: Receiver Operating Characteristic and Precision-Recall curves for all models. Joint training (joint-f2) achieves an AP of 0.92 and AUC of 0.96, outperforming the f2-f2 baseline (AP = 0.90, AUC = 0.94). The l7-f2 model shows lower generalization performance (AP = 0.78, AUC = 0.87), while l7-l7 reaches the highest scores overall (AP = 0.95, AUC = 0.98).