Efficient Adaptation of Deep Neural Networks for Semantic Segmentation in Space Applications

TL;DR

The paper tackles the challenge of deploying semantic rock segmentation in space under strict bandwidth and memory constraints. It introduces adapter-based transfer learning, integrated into a pre-trained backbone, and couples it with memory-saving strategies—adapter fusion and adapter ranking—to enable efficient domain adaptation for lunar and Martian landscapes. Through experiments on Moon and Mars datasets using U-Net backbones, the authors show that adapters can match the performance of full retraining with around 10% of the trained parameters, while ranking and fusion further reduce memory and computation with minimal performance loss. The work demonstrates practical implications for space missions, enabling on-board adaptation with limited data transmission and hardware resources. The findings pave the way for resource-aware, end-to-end deployable adaptation in autonomous space exploration systems.

Abstract

In recent years, the application of Deep Learning techniques has shown remarkable success in various computer vision tasks, paving the way for their deployment in extraterrestrial exploration. Transfer learning has emerged as a powerful strategy for addressing the scarcity of labeled data in these novel environments. This paper represents one of the first efforts in evaluating the feasibility of employing adapters toward efficient transfer learning for rock segmentation in extraterrestrial landscapes, mainly focusing on lunar and martian terrains. Our work suggests that the use of adapters, strategically integrated into a pre-trained backbone model, can be successful in reducing both bandwidth and memory requirements for the target extraterrestrial device. In this study, we considered two memory-saving strategies: layer fusion (to reduce to zero the inference overhead) and an ``adapter ranking'' (to also reduce the transmission cost). Finally, we evaluate these results in terms of task performance, memory, and computation on embedded devices, evidencing trade-offs that open the road to more research in the field.

Figures (14)

• Figure 1: Transmitting a fully fine-tuned model can be very bandwidth and memory-demanding. In this work, we explore the possibility of employing adapters, shallow modules that introduce corrections to a pre-loaded one (and that can be considered as an update).
• Figure 2: A schematic showing how adapters ($A_n$, in red) are plugged into the rest of the architecture (blue).
• Figure 3: Adapter fusion scheme, visualized step-by-step.
• Figure 4: Pareto curves for VGG19-BN and ResNet-18.
• Figure 5: Comparison of U-Net predictions on real Moon and real Mars images using different methods.