Promptable Foundation Models for SAR Remote Sensing: Adapting the Segment Anything Model for Snow Avalanche Segmentation

TL;DR

The paper addresses the challenge of snow avalanche segmentation in Sentinel-1 SAR imagery by adapting a segmentation foundation model (SAM) to SAR data. It develops a three-pronged approach: (i) domain adaptation via parameter-efficient adapters in the image encoder, (ii) multi-encoder input handling to exploit six-channel SAR data with a supervised alignment and a flexible fusion mechanism, and (iii) a robust prompt strategy plus an efficient training protocol, complemented by a semi-automatic annotation tool. The method achieves competitive automatic segmentation performance against baselines and delivers a substantial speedup (about $60\%$) in annotating SAR imagery, enabling scalable avalanche inventories. The work demonstrates practical impact for operational avalanche forecasting and provides a blueprint for extending foundation-model adaptations to other SAR-based detection tasks.

Abstract

Remote sensing solutions for avalanche segmentation and mapping are key to supporting risk forecasting and mitigation in mountain regions. Synthetic Aperture Radar (SAR) imagery from Sentinel-1 can be effectively used for this task, but training an effective detection model requires gathering a large dataset with high-quality annotations from domain experts, which is prohibitively time-consuming. In this work, we aim to facilitate and accelerate the annotation of SAR images for avalanche mapping. We build on the Segment Anything Model (SAM), a segmentation foundation model trained on natural images, and tailor it to Sentinel-1 SAR data. Adapting SAM to our use-case requires addressing several domain-specific challenges: (i) domain mismatch, since SAM was not trained on satellite/SAR imagery; (ii) input adaptation, because SAR products typically provide more than three channels, while SAM is constrained to RGB images; (iii) robustness to imprecise prompts that can affect target identification and degrade the segmentation quality, an issue exacerbated in small, low-contrast avalanches; and (iv) training efficiency, since standard fine-tuning is computationally demanding for SAM. We tackle these challenges through a combination of adapters to mitigate the domain gap, multiple encoders to handle multi-channel SAR inputs, prompt-engineering strategies to improve avalanche localization accuracy, and a training algorithm that limits the training time of the encoder, which is recognized as the major bottleneck. We integrate the resulting model into an annotation tool and show experimentally that it speeds up the annotation of SAR images.

Figures (11)

• Figure 1: Avalanche segmentation: (a,c) backscatter images created through Algorithm \ref{['alg:sar_rgb']} discussed in Appendix \ref{['manual_segmentation']} and (b,d) corresponding ground truth masks.
• Figure 2: Overview of the architecture. A heavyweight image encoder outputs an image embedding, given an RGB image. The prompt encoder identifies the segmentation target, which is then segmented by the mask decoder.
• Figure 3: Transformer block of the modified with adapters, composed of two linear layers and an activation function, positioned after the multi-head attention and in parallel with the .
• Figure 4: Creation of to improve robustness to inaccurate prompts. From left to right, we shown: the creation of the minimum enclosing from the segmentation mask, the random increase of the dimensions, and the merging of overlapping .
• Figure 5: sfg: given two image embeddings $e_1$ and $e_2$, the gate predicts weights $\omega$ from their concatenation and produces the fused embedding $\hat{e}_F$.