From Cold Start to Active Learning: Embedding-Based Scan Selection for Medical Image Segmentation

TL;DR

This work tackles the label-efficiency challenge in medical image segmentation by introducing a two-stage sampling framework: a cold-start phase that uses foundation-model embeddings, 2D t-SNE visualization, silhouette-guided clustering, medoid seeds, and budget-proportional augmentation; followed by an active learning phase that blends image-level uncertainty with spatial diversity in embedding space. The approach yields consistent improvements over random seeding and baseline clustering across 2D SynthStrip, Montgomery, and CheXmask-300 datasets, with additional gains when combining uncertainty and diversity during AL. The proposed method offers interpretable visualization of the sampling landscape and requires modest computational overhead, making it a practical baseline for low-data segmentation tasks. The results demonstrate that a principled balance of representational diversity and uncertainty enhances both overlap and boundary metrics, reducing variability and improving stability in data-scarce regimes. Overall, the framework provides a simple, effective, and transferable strategy for label-efficient medical image segmentation that can adapt to binary and multi-class tasks across modalities.

Abstract

Accurate segmentation annotations are critical for disease monitoring, yet manual labeling remains a major bottleneck due to the time and expertise required. Active learning (AL) alleviates this burden by prioritizing informative samples for annotation, typically through a diversity-based cold-start phase followed by uncertainty-driven selection. We propose a novel cold-start sampling strategy that combines foundation-model embeddings with clustering, including automatic selection of the number of clusters and proportional sampling across clusters, to construct a diverse and representative initial training. This is followed by an uncertainty-based AL framework that integrates spatial diversity to guide sample selection. The proposed method is intuitive and interpretable, enabling visualization of the feature-space distribution of candidate samples. We evaluate our approach on three datasets spanning X-ray and MRI modalities. On the CheXmask dataset, the cold-start strategy outperforms random selection, improving Dice from 0.918 to 0.929 and reducing the Hausdorff distance from 32.41 to 27.66 mm. In the AL setting, combined entropy and diversity selection improves Dice from 0.919 to 0.939 and reduces the Hausdorff distance from 30.10 to 19.16 mm. On the Montgomery dataset, cold-start gains are substantial, with Dice improving from 0.928 to 0.950 and Hausdorff distance decreasing from 14.22 to 9.38 mm. On the SynthStrip dataset, cold-start selection slightly affects Dice but reduces the Hausdorff distance from 9.43 to 8.69 mm, while active learning improves Dice from 0.816 to 0.826 and reduces the Hausdorff distance from 7.76 to 6.38 mm. Overall, the proposed framework consistently outperforms baseline methods in low-data regimes, improving segmentation accuracy.

Figures (8)

• Figure 1: Framework overview: cold-start followed by active selection
• Figure 2: Our cold-start method. First, foundation-model embeddings are extracted and projected to 2D using t-SNE; the optimal number of clusters is then selected via silhouette-scored k-means, cluster medoids are chosen as initial seeds, and the remaining budget is filled by farthest-point sampling within clusters to maximize diversity.
• Figure 3: Our combined active selection method. Entropy and diversity are computed separately and combined with a unified score.
• Figure 4: SynthStrip 2D Result Summary
• Figure 5: Illustrative Examples for results comparison between random selection and our active selection method that combined entropy with diversity. Large differences are emphasized with green and red circles.