Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CUTS: A Deep Learning and Topological Framework for Multigranular Unsupervised Medical Image Segmentation

Chen Liu, Matthew Amodio, Liangbo L. Shen, Feng Gao, Arman Avesta, Sanjay Aneja, Jay C. Wang, Lucian V. Del Priore, Smita Krishnaswamy

TL;DR

CUTS showed performance on par with Segment Anything Models (SAM, MedSAM, SAM-Med2D) pre-trained on gigantic labeled datasets, and improved the dice coefficient and Hausdorff distance by at least 10% compared to existing unsupervised methods.

Abstract

Segmenting medical images is critical to facilitating both patient diagnoses and quantitative research. A major limiting factor is the lack of labeled data, as obtaining expert annotations for each new set of imaging data and task can be labor intensive and inconsistent among annotators. We present CUTS, an unsupervised deep learning framework for medical image segmentation. CUTS operates in two stages. For each image, it produces an embedding map via intra-image contrastive learning and local patch reconstruction. Then, these embeddings are partitioned at dynamic granularity levels that correspond to the data topology. CUTS yields a series of coarse-to-fine-grained segmentations that highlight features at various granularities. We applied CUTS to retinal fundus images and two types of brain MRI images to delineate structures and patterns at different scales. When evaluated against predefined anatomical masks, CUTS improved the dice coefficient and Hausdorff distance by at least 10% compared to existing unsupervised methods. Finally, CUTS showed performance on par with Segment Anything Models (SAM, MedSAM, SAM-Med2D) pre-trained on gigantic labeled datasets.

CUTS: A Deep Learning and Topological Framework for Multigranular Unsupervised Medical Image Segmentation

TL;DR

CUTS showed performance on par with Segment Anything Models (SAM, MedSAM, SAM-Med2D) pre-trained on gigantic labeled datasets, and improved the dice coefficient and Hausdorff distance by at least 10% compared to existing unsupervised methods.

Abstract

Segmenting medical images is critical to facilitating both patient diagnoses and quantitative research. A major limiting factor is the lack of labeled data, as obtaining expert annotations for each new set of imaging data and task can be labor intensive and inconsistent among annotators. We present CUTS, an unsupervised deep learning framework for medical image segmentation. CUTS operates in two stages. For each image, it produces an embedding map via intra-image contrastive learning and local patch reconstruction. Then, these embeddings are partitioned at dynamic granularity levels that correspond to the data topology. CUTS yields a series of coarse-to-fine-grained segmentations that highlight features at various granularities. We applied CUTS to retinal fundus images and two types of brain MRI images to delineate structures and patterns at different scales. When evaluated against predefined anatomical masks, CUTS improved the dice coefficient and Hausdorff distance by at least 10% compared to existing unsupervised methods. Finally, CUTS showed performance on par with Segment Anything Models (SAM, MedSAM, SAM-Med2D) pre-trained on gigantic labeled datasets.
Paper Structure (29 sections, 6 equations, 4 figures, 1 table)

This paper contains 29 sections, 6 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Works
  3. Traditional methods for medical image segmentation
  4. Supervised learning for medical image segmentation
  5. Towards unsupervised learning
  6. Contrastive learning
  7. Unsupervised image segmentation with contrastive learning
  8. Segment Anything Model (SAM) and medical variants
  9. Methods
  10. Learning an embedding space for pixel-centered patches
  11. Intra-image contrastive loss
  12. Local patch reconstruction loss
  13. Final objective function
  14. Hyperparameters
  15. Coarse-graining for multiscale segmentation
  16. ...and 14 more sections

Figures (4)

  • Figure 1: The CUTS Framework. (A) Overview. (B) Pixel-centered patches are mapped into the embedding space, jointly optimized by two objectives. (C) Positive and negative patch pairs are selected based on proximity and structural similarity. (D) Diffusion condensation coarse grains embedding vectors at a series of granularities. (E) Segmentation for any granularity can be performed by mapping cluster assignments to the image space. Multiscale PHATE (MS-PHATE) MS_PHATE is used for visualization.
  • Figure 2: Effects of hyperparameters.
  • Figure 3: Multigranular segmentation (odd rows) captures distinctive patterns at various scales. Multiscale PHATE (even rows) is used to visualize the diffusion condensation process. The results of CUTS + spectral $k$-means clustering ("$k$-means") and CUTS + diffusion condensation persistent structures ("diffusion-P") are also shown for reference.
  • Figure 4: Qualitative segmentation comparison. Green curves outline the ground truth labels while blue or red curves outline the predictions. "diffusion-B": the best diffusion condensation granularity. "Sup.": supervised "P.T.": pre-training. "+bbox": using bounding boxes instead of points as input; included for completeness but would be unfair for comparison.