FALCON: Few-Shot Adversarial Learning for Cross-Domain Medical Image Segmentation

TL;DR

FALCON tackles the challenge of precise 3D medical image segmentation under limited annotations and privacy constraints by reframing the task as cross-domain few-shot segmentation on 2D slices. It meta-trains a lightweight encoder–relation module–decoder on natural images, then fine-tunes with boundary-aware Hausdorff loss and adversarial regularization using unlabeled intra-patient slices, followed by task-aware, gradient-free inference on new patients using unlabeled support. The framework achieves superior boundary accuracy (lowest 95th percentile Hausdorff Distance) and competitive Dice scores across four medical benchmarks while using far fewer labeled samples and far less compute (approximately 9.9M parameters and 2.3 GFLOPs) than state-of-the-art models. This makes FALCON well-suited for privacy-preserving, on-device deployment in annotation-scarce clinical settings, enabling accurate, patient-specific segmentation without extensive data collection or cloud-based processing. The study also demonstrates the value of unlabeled support as contextual priors and highlights the potential for cross-domain adaptability with minimal supervision in medical imaging.

Abstract

Precise delineation of anatomical and pathological structures within 3D medical volumes is crucial for accurate diagnosis, effective surgical planning, and longitudinal disease monitoring. Despite advancements in AI, clinically viable segmentation is often hindered by the scarcity of 3D annotations, patient-specific variability, data privacy concerns, and substantial computational overhead. In this work, we propose FALCON, a cross-domain few-shot segmentation framework that achieves high-precision 3D volume segmentation by processing data as 2D slices. The framework is first meta-trained on natural images to learn-to-learn generalizable segmentation priors, then transferred to the medical domain via adversarial fine-tuning and boundary-aware learning. Task-aware inference, conditioned on support cues, allows FALCON to adapt dynamically to patient-specific anatomical variations across slices. Experiments on four benchmarks demonstrate that FALCON consistently achieves the lowest Hausdorff Distance scores, indicating superior boundary accuracy while maintaining a Dice Similarity Coefficient comparable to the state-of-the-art models. Notably, these results are achieved with significantly less labeled data, no data augmentation, and substantially lower computational overhead.

Figures (6)

• Figure 1: Problem Formulation of Cross-Domain Few-Shot Segmentation (CDFSS). A model is trained on source tasks $\tau_s$, involving base classes $C_\text{base}$ from a source dataset $\mathrm{D}_s \sim \mathcal{D}$ (e.g., natural images). The objective is to generalize to target tasks $\tau_t$ involving previously unseen classes $C_\text{novel}$ from a distinct target dataset $\mathrm{D}_t \sim \mathcal{D}'$ (e.g., medical imaging). The underlying distributions for the source and the target dataset are denoted by $\mathcal{D}$ and $\mathcal{D}'$. This mimics human cognitive processes where medical trainees acquire broad foundational knowledge over time and later adapt it to specialize as clinicians. Unlike the source label-rich source domain, the target domain is characterized by limited data and scarce annotations.
• Figure 2: Overview of FALCON for precise boundary segmentation in medical imaging: (a) Training phase using abundant annotated natural images from the source domain $\mathrm{D}_{s}$, enabling the model to learn to learn segmentation knowledge. (b) In the next phase, Boundary-Aware Adversarial Fine-tuning (BAAF) adapts the model to the target medical domain $\mathrm{D}_{t}$ by leveraging a small annotated subset of slices along with a large collection of unlabeled slices as the support set for a patient $\pi$. (c) Test/Inference segments entire slices for a patient previously unseen during fine-tuning, while leveraging selective slices as an unlabeled support set, leading to task-aware inference. The segmentation network $f_\theta$ consists of three key components: an encoder ($E$), a relation module ($RM$), and a decoder ($D$).
• Figure 3: Datasets used in our experimental setup: (a) FSS-1000, a natural image dataset illustrated with 10 example classes; and the medical image datasets: (b) CHAOS-CT for liver segmentation, (c) Spleen CT for spleen segmentation, (d) COVID-19 CT for lung infection segmentation, and (e) Cardiac MRI for left atrium segmentation. Segmentation masks are shown in green for FSS-1000 and in red for the medical datasets.
• Figure 4: Distribution of unlabeled and labeled samples across the four medical imaging datasets used in this study. Each horizontal bar represents the number of samples categorized as unlabeled and labeled, further divided into training, validation, and test sets for the CHAOS-CT, Cardiac-MRI, Spleen-CT, and COVID-19 (CT) datasets. This visualization emphasizes the substantial presence of unlabeled data, approximately 60% in each dataset, highlighting the relevance of our approach for practical clinical scenarios where pixel-wise annotated data for segmentation is scarce. The number of samples is presented in $\mathrm{x}$-axis
• Figure 5: Qualitative results demonstrating the boundary delineation precision of our FALCON framework on the CHAOS-CT, Spleen-CT, COVID-19, and Cardiac-MRI datasets. Clinical ground truth annotations are shown in blue, while predicted segmentation maps are overlaid in red. Best viewed when zoomed in for clarity.