SAS: Segment Anything Small for Ultrasound -- A Non-Generative Data Augmentation Technique for Robust Deep Learning in Ultrasound Imaging

TL;DR

This work tackles the difficulty of segmenting small ultrasound structures by introducing SAS, a non-generative data augmentation technique that simultaneously alters scale and texture to enrich training diversity. By fine-tuning a promptable foundation model with iterative multi-click prompts, SAS achieves robust, generalizable segmentation across six internal/external datasets, with Dice score gains up to $0.35$ and significant improvements for small structures. The approach preserves accuracy for larger organs, avoids introducing artifacts, and reduces the need for extensive manual labeling, making it practical for data-constrained clinical settings. Overall, SAS enhances generalization to out-of-distribution anatomies and imaging conditions, offering a scalable, ethically sound augmentation strategy for ultrasound segmentation and potentially other imaging modalities.

Abstract

Accurate segmentation of anatomical structures in ultrasound (US) images, particularly small ones, is challenging due to noise and variability in imaging conditions (e.g., probe position, patient anatomy, tissue characteristics and pathology). To address this, we introduce Segment Anything Small (SAS), a simple yet effective scale- and texture-aware data augmentation technique designed to enhance the performance of deep learning models for segmenting small anatomical structures in ultrasound images. SAS employs a dual transformation strategy: (1) simulating diverse organ scales by resizing and embedding organ thumbnails into a black background, and (2) injecting noise into regions of interest to simulate varying tissue textures. These transformations generate realistic and diverse training data without introducing hallucinations or artifacts, improving the model's robustness to noise and variability. We fine-tuned a promptable foundation model on a controlled organ-specific medical imaging dataset and evaluated its performance on one internal and five external datasets. Experimental results demonstrate significant improvements in segmentation performance, with Dice score gains of up to 0.35 and an average improvement of 0.16 [95% CI 0.132,0.188]. Additionally, our iterative point prompts provide precise control and adaptive refinement, achieving performance comparable to bounding box prompts with just two points. SAS enhances model robustness and generalizability across diverse anatomical structures and imaging conditions, particularly for small structures, without compromising the accuracy of larger ones. By offering a computationally efficient solution that eliminates the need for extensive human labeling efforts, SAS emerges as a powerful tool for advancing medical image analysis, particularly in resource-constrained settings.

Figures (8)

• Figure 1: Segment Anything Small (SAS) image transformations. Step 1 - Generating thumbnails by resizing images to simulate small anatomical structures (left). Step 2 - Image perturbations inside the region of interest to represent varying pixel values and noise levels (right).
• Figure 2: Segmentation Performance: Iterative Point vs. Bounding Box Prompts. Comparison of segmentation performance using iterative point and bounding box prompts with full and partial fine-tuning. Dice scores are reported for iterative point prompts ranging from 1 to 10.
• Figure 3: Two-dimensional t-SNE visualization of feature embeddings for SAS-generated images, training set images, and test set images in Scenario 1 (left) and Scenario 2 (right). SAS-generated images in both scenarios total 20,000, while non-SAS images in Scenario 2 were downsampled to 20,000 for comparison. Test set details are provided in Table \ref{['tab:data']}. The embeddings were extracted using a VGG-16 encoder pre-trained on ImageNet-1K, with a t-SNE perplexity of 50.
• Figure 4: Evaluation of SAS. Normalized Surface Distance (NSD) and Dice Similarity Coefficient (DSC) results for all test datasets (see Table \ref{['tab:data']}) in two scenarios: a) Scenario 1 (orange lines), using a low-variety training dataset of kidney images, and b) Scenario 2 (blue lines), using a high-variety training dataset of gynecological images. Results are shown for models trained with and without SAS. Numbers 1 to 8 indicate the number of iterative click prompts used in the segmentation process. Confidence intervals (CIs) were bootstrapped 10,000 times. Asterisks (*) indicate that baseline non-SAS and SAS are not significantly different.
• Figure 5: Inference on Follicle and Breast Tumor - Violin plots showing the dice similarity coefficient and normalized surface distance scores for the small structures follicle and breast tumor datasets follicle (a-b) and BUSI (c-d) respectively. The training dataset for Scenario 1 consists of kidney images, while Scenario 2 uses gynecological images.