Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

USegMix: Unsupervised Segment Mix for Efficient Data Augmentation in Pathology Images

Jiamu Wang, Jin Tae Kwak

TL;DR

USegMix tackles data scarcity in pathology by a two-phase augmentation: Phase 1 builds a tissue segment pool through SAM-based unsupervised segmentation on $N_s=30$ SLIC superpixels, sampling $K=15$ prompts per superpixel to obtain anchor segments represented as $f_i\in\mathbb{R}^{128}$ after PCA; Phase 2 replaces a target segment with a semantically similar segment from the pool and employs a pre-trained latent diffusion model to inpaint and blend via forward and reverse SDEs, guided by the score function $s_{\theta}$ and enhanced by Manifold Constraint Gradient. The approach is evaluated on colorectal and prostate cancer datasets, training the LDM on patches from seven tissue types and synthesizing $600$ images per class; classifiers such as EfficientNet-B0, ResNet-50, DenseNet-121, and CTransPath show consistent improvements in accuracy, F1, and kappa over using original data alone. USegMix does not require additional labeling or target-dataset training, offering a practical, generic augmentation strategy for pathology and potentially whole-slide imaging, though limitations include incomplete gland coverage and the need to tailor the diffusion model to new domains. Overall, the method advances realistic data augmentation in histopathology, enabling more robust cancer diagnosis pipelines with limited labeled data.

Abstract

In computational pathology, researchers often face challenges due to the scarcity of labeled pathology datasets. Data augmentation emerges as a crucial technique to mitigate this limitation. In this study, we introduce an efficient data augmentation method for pathology images, called USegMix. Given a set of pathology images, the proposed method generates a new, synthetic image in two phases. In the first phase, USegMix constructs a pool of tissue segments in an automated and unsupervised manner using superpixels and the Segment Anything Model (SAM). In the second phase, USegMix selects a candidate segment in a target image, replaces it with a similar segment from the segment pool, and blends them by using a pre-trained diffusion model. In this way, USegMix can generate diverse and realistic pathology images. We rigorously evaluate the effectiveness of USegMix on two pathology image datasets of colorectal and prostate cancers. The results demonstrate improvements in cancer classification performance, underscoring the substantial potential of USegMix for pathology image analysis.

USegMix: Unsupervised Segment Mix for Efficient Data Augmentation in Pathology Images

TL;DR

USegMix tackles data scarcity in pathology by a two-phase augmentation: Phase 1 builds a tissue segment pool through SAM-based unsupervised segmentation on SLIC superpixels, sampling prompts per superpixel to obtain anchor segments represented as after PCA; Phase 2 replaces a target segment with a semantically similar segment from the pool and employs a pre-trained latent diffusion model to inpaint and blend via forward and reverse SDEs, guided by the score function and enhanced by Manifold Constraint Gradient. The approach is evaluated on colorectal and prostate cancer datasets, training the LDM on patches from seven tissue types and synthesizing images per class; classifiers such as EfficientNet-B0, ResNet-50, DenseNet-121, and CTransPath show consistent improvements in accuracy, F1, and kappa over using original data alone. USegMix does not require additional labeling or target-dataset training, offering a practical, generic augmentation strategy for pathology and potentially whole-slide imaging, though limitations include incomplete gland coverage and the need to tailor the diffusion model to new domains. Overall, the method advances realistic data augmentation in histopathology, enabling more robust cancer diagnosis pipelines with limited labeled data.

Abstract

In computational pathology, researchers often face challenges due to the scarcity of labeled pathology datasets. Data augmentation emerges as a crucial technique to mitigate this limitation. In this study, we introduce an efficient data augmentation method for pathology images, called USegMix. Given a set of pathology images, the proposed method generates a new, synthetic image in two phases. In the first phase, USegMix constructs a pool of tissue segments in an automated and unsupervised manner using superpixels and the Segment Anything Model (SAM). In the second phase, USegMix selects a candidate segment in a target image, replaces it with a similar segment from the segment pool, and blends them by using a pre-trained diffusion model. In this way, USegMix can generate diverse and realistic pathology images. We rigorously evaluate the effectiveness of USegMix on two pathology image datasets of colorectal and prostate cancers. The results demonstrate improvements in cancer classification performance, underscoring the substantial potential of USegMix for pathology image analysis.

Paper Structure

This paper contains 18 sections, 4 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methods
  3. USegMix Phase 1 - Segment Pool Preparation
  4. SAM-based Unsupervised Segmentation.
  5. Feature Representation.
  6. USegMix Phase 2 - Augmentation by Segment Replacement
  7. Segment Replacement.
  8. Diffusion Model Inpainting.
  9. Experiments
  10. Datasets
  11. Diffusion Model Training Dataset.
  12. Cancer Classification Dataset.
  13. Implementation Details
  14. Results
  15. Image Synthesis.
  16. ...and 3 more sections

Figures (5)

  • Figure 1: Comparison among USegMix(ours) and CutOut, MixUp, CutMix.
  • Figure 2: The two-phase procedure of USegMix. In phase 1, it first conducts SAM-based unsupervised segmentation to produce the segmentation map for each image, extracts features for each segment, and builds a tissue segment pool. In phase 2, it randomly selects a segment (Segment A) to replace, selects a similar segment (Segment B) from the segment pool, replaces Segment A with Segment B, and conducts inpainting using a pre-trained diffusion model to enhance the realism of the resultant image.
  • Figure 3: The procedure of SAM-based unsupervised segmentation in USegMix phase 1. Given an image, it generates superpixels, selects a target superpixel, samples a random point within the target superpixel, feeds SAM with the random point as a prompt for K times, and collects the resultant masks from SAM. Then, it clusters the resultant K masks based on their similarity and selects the most prevalent mask as the anchor mask to represent the superpixel.
  • Figure 4: Examples of unsupervised segmentation (USegMix Phase 1). (a) Colorectal and (b) prostate tissues.
  • Figure 5: Examples of segment replacement and inpainting (USegMix Phase 2). (a) Colorectal and (b) prostate tissues.