Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Large-Scale Label Quality Assessment for Medical Segmentation via a Vision-Language Judge and Synthetic Data

Yixiong Chen, Zongwei Zhou, Wenxuan Li, Alan Yuille

TL;DR

We address label quality in large-scale medical segmentation datasets by introducing SegAE, a vision-language QC model trained on synthetic label-quality pairs to predict DSC between pseudo-labels and ground truth. SegAE fuses vision features with class text embeddings via a class-conditioned attention and trains with a MSE loss plus an optimal pair ranking loss; label-quality data are synthesized from STUNet on a 142-organ CT dataset, yielding over 4 million slice–mask pairs. It achieves a strong correlation with ground-truth DSC (r=0.902) and a per-label evaluation time of ~0.06 s, and it improves data efficiency for active and semi-supervised training while revealing distribution-wide label-quality issues. The work provides a scalable QC tool for large datasets, with public data, model weights, and code to facilitate adoption.

Abstract

Large-scale medical segmentation datasets often combine manual and pseudo-labels of uneven quality, which can compromise training and evaluation. Low-quality labels may hamper performance and make the model training less robust. To address this issue, we propose SegAE (Segmentation Assessment Engine), a lightweight vision-language model (VLM) that automatically predicts label quality across 142 anatomical structures. Trained on over four million image-label pairs with quality scores, SegAE achieves a high correlation coefficient of 0.902 with ground-truth Dice similarity and evaluates a 3D mask in 0.06s. SegAE shows several practical benefits: (I) Our analysis reveals widespread low-quality labeling across public datasets; (II) SegAE improves data efficiency and training performance in active and semi-supervised learning, reducing dataset annotation cost by one-third and quality-checking time by 70% per label. This tool provides a simple and effective solution for quality control in large-scale medical segmentation datasets. The dataset, model weights, and codes are released at https://github.com/Schuture/SegAE.

Large-Scale Label Quality Assessment for Medical Segmentation via a Vision-Language Judge and Synthetic Data

TL;DR

We address label quality in large-scale medical segmentation datasets by introducing SegAE, a vision-language QC model trained on synthetic label-quality pairs to predict DSC between pseudo-labels and ground truth. SegAE fuses vision features with class text embeddings via a class-conditioned attention and trains with a MSE loss plus an optimal pair ranking loss; label-quality data are synthesized from STUNet on a 142-organ CT dataset, yielding over 4 million slice–mask pairs. It achieves a strong correlation with ground-truth DSC (r=0.902) and a per-label evaluation time of ~0.06 s, and it improves data efficiency for active and semi-supervised training while revealing distribution-wide label-quality issues. The work provides a scalable QC tool for large datasets, with public data, model weights, and code to facilitate adoption.

Abstract

Large-scale medical segmentation datasets often combine manual and pseudo-labels of uneven quality, which can compromise training and evaluation. Low-quality labels may hamper performance and make the model training less robust. To address this issue, we propose SegAE (Segmentation Assessment Engine), a lightweight vision-language model (VLM) that automatically predicts label quality across 142 anatomical structures. Trained on over four million image-label pairs with quality scores, SegAE achieves a high correlation coefficient of 0.902 with ground-truth Dice similarity and evaluates a 3D mask in 0.06s. SegAE shows several practical benefits: (I) Our analysis reveals widespread low-quality labeling across public datasets; (II) SegAE improves data efficiency and training performance in active and semi-supervised learning, reducing dataset annotation cost by one-third and quality-checking time by 70% per label. This tool provides a simple and effective solution for quality control in large-scale medical segmentation datasets. The dataset, model weights, and codes are released at https://github.com/Schuture/SegAE.
Paper Structure (17 sections, 3 equations, 5 figures, 3 tables)

This paper contains 17 sections, 3 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Segmentation Assessment Engine
  3. Formulation of the Framework
  4. SegAE Architecture
  5. Optimal Pair Ranking Loss
  6. Label Quality Data Synthesis
  7. Experiments & Results
  8. Experimental Setting
  9. Evaluation of SegAE
  10. Ablation Study.
  11. Evaluation of Model Configurations.
  12. External Evaluation.
  13. Dataset Benchmarking with SegAE
  14. Data-Efficient Training
  15. Conclusion
  16. ...and 2 more sections

Figures (5)

  • Figure 1: Overview of SegAE: SegAE employs a vision encoder to encode an image with its visualized segmentation mask, and a language encoder to encode the text description of a target class. It predicts DSC relative to ground truths. Training is performed on synthetic masks generated from segmentation checkpoints.
  • Figure 2: Illustration of (a) the curves of model LCC/SROCC and data amount and (b) the curves of performance deviation w.r.t. #slice compared to prediction with all slices.
  • Figure 3: The scatter plots between predicted DSC and ground truth DSC of (a) SegAE testing set and (b, c) external evaluation on BTCV with two degenerations.
  • Figure 4: Illustration of samples with bad (left) and good (right) labels in AbdomenAtlas dataset.
  • Figure 5: Comparison of QC methods on active learning.