Rapid Distributed Fine-tuning of a Segmentation Model Onboard Satellites

TL;DR

The findings show that MobileSAM can be rapidly fine-tuned and benefits from decentralised learning, considering the constraints imposed by the simulated orbital environment, and improvements in segmentation performance with minimal training data and fast fine-tuning when satellites frequently communicate model updates are observed.

Abstract

Segmentation of Earth observation (EO) satellite data is critical for natural hazard analysis and disaster response. However, processing EO data at ground stations introduces delays due to data transmission bottlenecks and communication windows. Using segmentation models capable of near-real-time data analysis onboard satellites can therefore improve response times. This study presents a proof-of-concept using MobileSAM, a lightweight, pre-trained segmentation model, onboard Unibap iX10-100 satellite hardware. We demonstrate the segmentation of water bodies from Sentinel-2 satellite imagery and integrate MobileSAM with PASEOS, an open-source Python module that simulates satellite operations. This integration allows us to evaluate MobileSAM's performance under simulated conditions of a satellite constellation. Our research investigates the potential of fine-tuning MobileSAM in a decentralised way onboard multiple satellites in rapid response to a disaster. Our findings show that MobileSAM can be rapidly fine-tuned and benefits from decentralised learning, considering the constraints imposed by the simulated orbital environment. We observe improvements in segmentation performance with minimal training data and fast fine-tuning when satellites frequently communicate model updates. This study contributes to the field of onboard AI by emphasising the benefits of decentralised learning and fine-tuning pre-trained models for rapid response scenarios. Our work builds on recent related research at a critical time; as extreme weather events increase in frequency and magnitude, rapid response with onboard data analysis is essential.

Figures (8)

• Figure 1: Example of a 256x256 satellite tile and corresponding ground truth flood mask from the WorldFloods dataset, with a bounding box.
• Figure 2: MobileSAM model architecture. Image adapted from Zhang_Han_Qiao_Kim_Bae_Lee_Hong_2023.
• Figure 3: Flow chart showing the PASEOS decision-making process.
• Figure 4: Segmentation performance of the MobileSAM model before fine-tuning. Images show water body samples from the disaster site test location at Ylitornio.
• Figure 5: Cumulative frequency of IoU values for a satellite from each scenario.