SAM-Aug: Leveraging SAM Priors for Few-Shot Parcel Segmentation in Satellite Time Series

TL;DR

SAM-Aug tackles the challenge of few-shot, time-series land cover segmentation in remote sensing by introducing geometry-aware priors from the Segment Anything Model (SAM). The framework constructs cloud-free composites and uses SAM-derived region masks as priors, regularized through RegionSmoothLoss to enforce temporal-consistency within regions without fine-tuning SAM. Empirical results on the PASTIS-R dataset under a 5% labeling regime show a mean test mIoU of 36.21%, a relative improvement of 6.89% over the state-of-the-art baseline, with the best seed reaching 40.28% mIoU. The approach provides an annotation-efficient, plug-and-play regularization mechanism that leverages foundation model priors to improve land cover mapping in data-scarce settings, with potential extensions to panoptic segmentation and multi-modal data.

Abstract

Few-shot semantic segmentation of time-series remote sensing images remains a critical challenge, particularly in regions where labeled data is scarce or costly to obtain. While state-of-the-art models perform well under full supervision, their performance degrades significantly under limited labeling, limiting their real-world applicability. In this work, we propose SAM-Aug, a new annotation-efficient framework that leverages the geometry-aware segmentation capability of the Segment Anything Model (SAM) to improve few-shot land cover mapping. Our approach constructs cloud-free composite images from temporal sequences and applies SAM in a fully unsupervised manner to generate geometry-aware mask priors. These priors are then integrated into training through a proposed loss function called RegionSmoothLoss, which enforces prediction consistency within each SAM-derived region across temporal frames, effectively regularizing the model to respect semantically coherent structures. Extensive experiments on the PASTIS-R benchmark under a 5 percent labeled setting demonstrate the effectiveness and robustness of SAM-Aug. Averaged over three random seeds (42, 2025, 4090), our method achieves a mean test mIoU of 36.21 percent, outperforming the state-of-the-art baseline by +2.33 percentage points, a relative improvement of 6.89 percent. Notably, on the most favorable split (seed=42), SAM-Aug reaches a test mIoU of 40.28 percent, representing an 11.2 percent relative gain with no additional labeled data. The consistent improvement across all seeds confirms the generalization power of leveraging foundation model priors under annotation scarcity. Our results highlight that vision models like SAM can serve as useful regularizers in few-shot remote sensing learning, offering a scalable and plug-and-play solution for land cover monitoring without requiring manual annotations or model fine-tuning.

Figures (6)

• Figure 1: Overview of the proposed SAM-Aug framework. A cloud-free composite is generated from multi-temporal inputs, SAM extracts region masks, and RegionSmoothLoss enforces prediction consistency within each region across time.
• Figure 2: Per-class mIoU comparison across three random seeds (42, 2025, 4090). Each bar represents the mIoU for one land cover class on the test set. Gray bars: Baseline (SOTA); Colored bars: SAM-Aug (Ours). Our method shows consistent improvements, especially on challenging classes such as Sunflower, Leguminous fodder, and Mixed cereal.
• Figure 3: Performance comparison of SAM-Aug and SOTA across different training ratios under seed of 42. (a) Test mIoU shows gains at 5%--10% labeling. (b) Test OA follows a similar trend. (c) The validation-test gap is smaller for SAM-Aug at 5% and 7%, indicating better generalization.
• Figure 4: Training and validation loss curves under varying training data ratios. As the amount of labeled data increases, the loss converges faster, with reduced oscillations and smaller generalization gaps, indicating improved model stability and generalization.
• Figure 5: Training dynamics of the proposed RegionSmoothLoss. Left y-axis: absolute values of TotalLoss (gray) and loss_structure (blue). Right y-axis: relative contribution in percent (red, dashed). Despite its small magnitude, the RegionSmoothLoss provides consistent regularization throughout training.