Scale-aware Adaptive Supervised Network with Limited Medical Annotations

TL;DR

The paper tackles limited annotations and inter-annotator variability in medical image segmentation by introducing SASNet, a dual-branch network that jointly leverages low- and high-level features. It integrates a Scale-Aware Adaptive Reweighting strategy and a View Variance Enhancement mechanism, coupled with Segmentation-Regression Consistency via Signed Distance Maps, to robustly fuse multi-scale predictions and align segmentation with geometric priors. The approach achieves state-of-the-art results on LA, Pancreas-CT, and BraTS under limited labels, often approaching or surpassing fully supervised performance, and demonstrates strong ablations across components. The work provides a practical, open-source solution for semi-supervised medical segmentation with clear benefits for boundary refinement and small-structure detection, with potential applicability across modalities and tasks.

Abstract

Medical image segmentation faces critical challenges in semi-supervised learning scenarios due to severe annotation scarcity requiring expert radiological knowledge, significant inter-annotator variability across different viewpoints and expertise levels, and inadequate multi-scale feature integration for precise boundary delineation in complex anatomical structures. Existing semi-supervised methods demonstrate substantial performance degradation compared to fully supervised approaches, particularly in small target segmentation and boundary refinement tasks. To address these fundamental challenges, we propose SASNet (Scale-aware Adaptive Supervised Network), a dual-branch architecture that leverages both low-level and high-level feature representations through novel scale-aware adaptive reweight mechanisms. Our approach introduces three key methodological innovations, including the Scale-aware Adaptive Reweight strategy that dynamically weights pixel-wise predictions using temporal confidence accumulation, the View Variance Enhancement mechanism employing 3D Fourier domain transformations to simulate annotation variability, and segmentation-regression consistency learning through signed distance map algorithms for enhanced boundary precision. These innovations collectively address the core limitations of existing semi-supervised approaches by integrating spatial, temporal, and geometric consistency principles within a unified optimization framework. Comprehensive evaluation across LA, Pancreas-CT, and BraTS datasets demonstrates that SASNet achieves superior performance with limited labeled data, surpassing state-of-the-art semi-supervised methods while approaching fully supervised performance levels. The source code for SASNet is available at https://github.com/HUANGLIZI/SASNet.

Figures (8)

• Figure 1: Comparison of segmentation results between different branch networks and multi-view inputs in the dual-branch network. The dual-branch network structure performs better in some details and is closer to the Ground Truth values than the low-level branch and high-level branch approach. Additionally, our model exhibits distinct prediction styles across different views, akin to the natural variability among annotations.
• Figure 2: Overview of SASNet. SASNet consists of three key components: the Dual-branch Architectural Network, the View Variance Enhancement Mechanism, and the Scale-Aware Adaptive Reweight Strategy. FT and IFT represent 3D Fourier Transform and 3D Inverse Fourier Transform, respectively. The training of SASNet is under the supervision of PLC Loss, SRC Loss, and SEG Loss.
• Figure 3: Details of the Scale-Aware Adaptive Reweight (SAR) strategy at the i-th epoch.
• Figure 4: Diagram of Scale-aware Adaptive Reweight (SAR). Comparison of output results with and without SAR. Assuming that the predicted probabilities $Prob_{l}$ from the low-level branch and $Prob_{h}$ from the high-level branch remain unchanged. $C_{1}^{l}$ stands for the confidence matrix of the low-level branch in the first epoch. $\alpha$ represents the residual coefficient. Weighted $Prob_{l}$ and Weighted $Prob_{h}$ represent the probabilities after weighting for the low-level branch and high-level branch, respectively. $C_{ch1}^{l}$ denotes the first channel of $C_{1}^{l}$, while $C_{ch2}^{l}$ signifies the second channel. Similarly, $C_{ch1}^{h}$ and $C_{ch2}^{h}$ stand for the first and second channels of $C_{1}^{h}$, respectively.
• Figure 5: Diagram of SDM (Signed Distance Map) Generation Pipeline. The workflow includes boundary extraction via feedback (FB), parallel distance transforms (DT and $\sim$DT) producing positive and negative distance maps.