Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Auto-Generating Weak Labels for Real & Synthetic Data to Improve Label-Scarce Medical Image Segmentation

Tanvi Deshpande, Eva Prakash, Elsie Gyang Ross, Curtis Langlotz, Andrew Ng, Jeya Maria Jose Valanarasu

TL;DR

This work tackles label scarcity in medical image segmentation by coupling a small gold-standard set with a coarse-label predictor to auto-prompt MedSAM and generate weak labels on unlabeled data. The generated weak labels are used to augment training data, enabling a more accurate segmentation model without extensive expert labeling. The approach is validated across ultrasound, dermoscopy, and chest X-ray tasks, and is further extended with synthetic data generated by diffusion models, demonstrating robust improvements in $DICE$ scores and demonstrating the practical potential for scalable, label-efficient medical imaging pipelines. MedSAM outperforms SAM in auto-prompting, reducing the need for manual intervention and enabling automatic label generation for real and synthetic data, with implications for human-in-the-loop annotation and deployment in label-scarce settings.

Abstract

The high cost of creating pixel-by-pixel gold-standard labels, limited expert availability, and presence of diverse tasks make it challenging to generate segmentation labels to train deep learning models for medical imaging tasks. In this work, we present a new approach to overcome the hurdle of costly medical image labeling by leveraging foundation models like Segment Anything Model (SAM) and its medical alternate MedSAM. Our pipeline has the ability to generate weak labels for any unlabeled medical image and subsequently use it to augment label-scarce datasets. We perform this by leveraging a model trained on a few gold-standard labels and using it to intelligently prompt MedSAM for weak label generation. This automation eliminates the manual prompting step in MedSAM, creating a streamlined process for generating labels for both real and synthetic images, regardless of quantity. We conduct experiments on label-scarce settings for multiple tasks pertaining to modalities ranging from ultrasound, dermatology, and X-rays to demonstrate the usefulness of our pipeline. The code is available at https://github.com/stanfordmlgroup/Auto-Generate-WLs/.

Auto-Generating Weak Labels for Real & Synthetic Data to Improve Label-Scarce Medical Image Segmentation

TL;DR

This work tackles label scarcity in medical image segmentation by coupling a small gold-standard set with a coarse-label predictor to auto-prompt MedSAM and generate weak labels on unlabeled data. The generated weak labels are used to augment training data, enabling a more accurate segmentation model without extensive expert labeling. The approach is validated across ultrasound, dermoscopy, and chest X-ray tasks, and is further extended with synthetic data generated by diffusion models, demonstrating robust improvements in scores and demonstrating the practical potential for scalable, label-efficient medical imaging pipelines. MedSAM outperforms SAM in auto-prompting, reducing the need for manual intervention and enabling automatic label generation for real and synthetic data, with implications for human-in-the-loop annotation and deployment in label-scarce settings.

Abstract

The high cost of creating pixel-by-pixel gold-standard labels, limited expert availability, and presence of diverse tasks make it challenging to generate segmentation labels to train deep learning models for medical imaging tasks. In this work, we present a new approach to overcome the hurdle of costly medical image labeling by leveraging foundation models like Segment Anything Model (SAM) and its medical alternate MedSAM. Our pipeline has the ability to generate weak labels for any unlabeled medical image and subsequently use it to augment label-scarce datasets. We perform this by leveraging a model trained on a few gold-standard labels and using it to intelligently prompt MedSAM for weak label generation. This automation eliminates the manual prompting step in MedSAM, creating a streamlined process for generating labels for both real and synthetic images, regardless of quantity. We conduct experiments on label-scarce settings for multiple tasks pertaining to modalities ranging from ultrasound, dermatology, and X-rays to demonstrate the usefulness of our pipeline. The code is available at https://github.com/stanfordmlgroup/Auto-Generate-WLs/.
Paper Structure (23 sections, 6 figures, 5 tables)

This paper contains 23 sections, 6 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Method
  4. Preliminaries
  5. Coarse Label Generation
  6. Input Selection
  7. Labeling and Filtering
  8. Augmenting the training data
  9. Synthetic Data
  10. Experiments and Results
  11. Datasets
  12. Implementation Details
  13. Results
  14. Synthetic Data
  15. Ablation Studies
  16. ...and 8 more sections

Figures (6)

  • Figure 1: Labels obtained from different configurations of SAM and MedSAM. Our method auto-generates effective input prompts (bounding boxes) using only very few annotations to generate high quality weak labels while SAM and MedSAM fail in auto options and are sensitive to input prompts in manual option.
  • Figure 2: An illustration of our pipeline to auto-generate weak labels for unlabeled data. We use the limited annotations to train an initial model that generates low-quality coarse labels on unlabeled data. We then select inputs from these coarse labels as prompts to MedSAM to create higher-quality weak labels. These weak labels are used to train a stronger segmentation model.
  • Figure 3: Example of prompting and labeling from coarse label.
  • Figure 4: Example outputs from each dataset
  • Figure 5: Comparison to SAM and MedSAM automatic segmentation for each dataset
  • ...and 1 more figures