Adaptive Attention Distillation for Robust Few-Shot Segmentation under Environmental Perturbations

TL;DR

This work tackles the limited robustness of few-shot segmentation under real-world environmental perturbations by introducing the Environment-Robust FSS (ER-FSS) benchmark and the Adaptive Attention Distillation (AAD) method. ER-FSS provides eight diverse datasets with challenging query samples to evaluate environmental resilience, while AAD distills class-specific attention through iterative, cross-scale interactions between support and query features using foreground masks and a compact set of learnable class queries. The combination of a correlation-based coarse localization (AMMs) and the foreground-driven distillation (AQGs) yields a robust, multi-scale segmentation framework that significantly outperforms state-of-the-art methods across 1-, 5-, and higher-shot settings. The findings demonstrate improved generalization and practical applicability for robust segmentation in surveillance, agricultural monitoring, and industrial inspection under realistic environmental conditions.

Abstract

Few-shot segmentation (FSS) aims to rapidly learn novel class concepts from limited examples to segment specific targets in unseen images, and has been widely applied in areas such as medical diagnosis and industrial inspection. However, existing studies largely overlook the complex environmental factors encountered in real world scenarios-such as illumination, background, and camera viewpoint-which can substantially increase the difficulty of test images. As a result, models trained under laboratory conditions often fall short of practical deployment requirements. To bridge this gap, in this paper, an environment-robust FSS setting is introduced that explicitly incorporates challenging test cases arising from complex environments-such as motion blur, small objects, and camouflaged targets-to enhance model's robustness under realistic, dynamic conditions. An environment robust FSS benchmark (ER-FSS) is established, covering eight datasets across multiple real world scenarios. In addition, an Adaptive Attention Distillation (AAD) method is proposed, which repeatedly contrasts and distills key shared semantics between known (support) and unknown (query) images to derive class-specific attention for novel categories. This strengthens the model's ability to focus on the correct targets in complex environments, thereby improving environmental robustness. Comparative experiments show that AAD improves mIoU by 3.3% - 8.5% across all datasets and settings, demonstrating superior performance and strong generalization. The source code and dataset are available at: https://github.com/guoqianyu-alberta/Adaptive-Attention-Distillation-for-FSS.

Figures (5)

• Figure 1: The overview of this paper comprises: (a) a comparison of environmental difficulty between query and support images in Environment-Robust Few-Shot Segmentation; (b) the proposed Environment-Robust FSS Benchmark (ER-FSS), including its covered scenarios and datasets; (c) a comparison between the proposed Adaptive Attention Distillation (AAD) method (I) and existing approaches (pre-training segmentation methods (I) and traditional few-shot segmentation (FSS) methods (II)); and (d) comparative experimental results demonstrating the methodological advancement.
• Figure 2: The construction process of Environment-robust Few-shot Segmentation (ER-FSS) Benchmark: data collection phase (a) and manual annotation phase (b). Data statistics (c) and a visualization example with t-SNE for the evaluation benchmark datasets.
• Figure 3: The pipeline of the proposed Adaptive Attention Distillation (AAD) framework. AAD framework consists of four parts: the encoder, correlation leaner module, adaptive attention distillation learner, and the decoder.
• Figure 4: Visualization of query maps at different stages with Swin-transformer backbone. After CL refers to the output after the correlation learning module.
• Figure 5: Comparison of segmentation results between our method and SOTA methods on various evaluation datasets and under multiple difficult scenarios.