Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Mix for Semi-Supervised Medical Image Segmentation

Zhiqiang Shen, Peng Cao, Junming Su, Jinzhu Yang, Osmar R. Zaiane

TL;DR

This work tackles label scarcity in medical image segmentation by refining consistency-regularization through adaptive mix-up. The authors introduce AdaMix, a self-paced perturbation strategy that starts with easier, high-confidence regions and progressively incorporates harder, boundary-focused content, parameterized by a self-paced mask and self-paced weight. Integrated into self-training, mean-teacher, and co-training (yielding AdaMix-ST, AdaMix-MT, AdaMix-CT), AdaMix achieves state-of-the-art results across ACDC, LA, and ISIC datasets, with notable improvements in Dice similarity and boundary metrics. The findings highlight that perturbation strategy, guided by the model’s learning state, is a critical driver of SSL performance in medical image segmentation, potentially reducing the need for extensive labeled data.

Abstract

Mix-up is a key technique for consistency regularization-based semi-supervised learning methods, blending two or more images to generate strong-perturbed samples for strong-weak pseudo supervision. Existing mix-up operations are performed either randomly or with predefined fixed rules, such as replacing low-confidence patches with high-confidence ones. The former lacks control over the perturbation degree, leading to overfitting on randomly perturbed samples, while the latter tends to generate images with trivial perturbations, both of which limit the effectiveness of consistency regularization. This paper aims to answer the following question: How can image mix-up perturbation be adaptively performed during training? To this end, we propose an Adaptive Mix algorithm (AdaMix) for image mix-up in a self-paced learning manner. Given that, in general, a model's performance gradually improves during training, AdaMix is equipped with a self-paced curriculum that, in the initial training stage, provides relatively simple perturbed samples and then gradually increases the difficulty of perturbed images by adaptively controlling the perturbation degree based on the model's learning state estimated by a self-paced regularize. We develop three frameworks with our AdaMix, i.e., AdaMix-ST, AdaMix-MT, and AdaMix-CT, for semi-supervised medical image segmentation. Extensive experiments on three public datasets show that the proposed frameworks can achieve superior performance. For example, compared with the state-of-the-art, AdaMix-CT achieves relative improvements of 2.62% in Dice similarity coefficient and 48.25% in average surface distance on the ACDC dataset with 10% labeled data. The results demonstrate that mix-up operations with dynamically adjusted perturbation strength based on the segmentation model's state can significantly enhance the effectiveness of consistency regularization.

Adaptive Mix for Semi-Supervised Medical Image Segmentation

TL;DR

This work tackles label scarcity in medical image segmentation by refining consistency-regularization through adaptive mix-up. The authors introduce AdaMix, a self-paced perturbation strategy that starts with easier, high-confidence regions and progressively incorporates harder, boundary-focused content, parameterized by a self-paced mask and self-paced weight. Integrated into self-training, mean-teacher, and co-training (yielding AdaMix-ST, AdaMix-MT, AdaMix-CT), AdaMix achieves state-of-the-art results across ACDC, LA, and ISIC datasets, with notable improvements in Dice similarity and boundary metrics. The findings highlight that perturbation strategy, guided by the model’s learning state, is a critical driver of SSL performance in medical image segmentation, potentially reducing the need for extensive labeled data.

Abstract

Mix-up is a key technique for consistency regularization-based semi-supervised learning methods, blending two or more images to generate strong-perturbed samples for strong-weak pseudo supervision. Existing mix-up operations are performed either randomly or with predefined fixed rules, such as replacing low-confidence patches with high-confidence ones. The former lacks control over the perturbation degree, leading to overfitting on randomly perturbed samples, while the latter tends to generate images with trivial perturbations, both of which limit the effectiveness of consistency regularization. This paper aims to answer the following question: How can image mix-up perturbation be adaptively performed during training? To this end, we propose an Adaptive Mix algorithm (AdaMix) for image mix-up in a self-paced learning manner. Given that, in general, a model's performance gradually improves during training, AdaMix is equipped with a self-paced curriculum that, in the initial training stage, provides relatively simple perturbed samples and then gradually increases the difficulty of perturbed images by adaptively controlling the perturbation degree based on the model's learning state estimated by a self-paced regularize. We develop three frameworks with our AdaMix, i.e., AdaMix-ST, AdaMix-MT, and AdaMix-CT, for semi-supervised medical image segmentation. Extensive experiments on three public datasets show that the proposed frameworks can achieve superior performance. For example, compared with the state-of-the-art, AdaMix-CT achieves relative improvements of 2.62% in Dice similarity coefficient and 48.25% in average surface distance on the ACDC dataset with 10% labeled data. The results demonstrate that mix-up operations with dynamically adjusted perturbation strength based on the segmentation model's state can significantly enhance the effectiveness of consistency regularization.
Paper Structure (30 sections, 1 theorem, 7 equations, 9 figures, 4 tables, 2 algorithms)

This paper contains 30 sections, 1 theorem, 7 equations, 9 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Semi-Supervised Learning
  4. Mix-up Augmentation
  5. Medical Image Segmentation
  6. Method
  7. Problem statement
  8. Overview
  9. Adaptive Mix Framework
  10. Adaptive Mix algorithm
  11. Self-Paced Mask
  12. Self-Paced Weight
  13. Experiments and Results
  14. Datasets
  15. Implementation Details
  16. ...and 15 more sections

Key Result

Theorem 1

Convergence theorem. Using the stochastic gradient descent with a learning rate $\eta_t = \frac{\eta_0}{t}$ at the $t^{th}$ iteration ($\eta_0$ refers to the initial learning rate) and the total optimization steps $T$, the convergence rate satisfies: where the term $\frac{\eta_0\ell\sigma^2}{\sqrt{T}}$ dominates the bound for large $T$.

Figures (9)

  • Figure 1: Comparison among CutMix, UMix shen2023co, inverse UMix, and our AdaMix. (a) Original and auxiliary images; (b) CutMix randomly replaces patches of the original image with patches from the auxiliary image, with an uncontrollable perturbation degree; (c) UMix replaces low-confidence patches in the original image with high-confidence ones from the auxiliary image, often generating perturbed examples with trivial perturbations; (d) I-UMix mixes high-confidence patches in the original image with low-confidence ones from the auxiliary image, yielding overly strong-perturbed images; (e) Our AdaMix synthesizes images with more high-confidence regions in the initial training stage and then gradually increases the perturbation degree to enhance the complexity of the perturbed images as training progresses; (f) The unsupervised learning loss curves; (g) The validation loss curves (and the Dice Similarity Coefficients on the ACDC test set).
  • Figure 2: Overview of the proposed Adaptive Mix framework for semi-supervised medical image segmentation. It includes our Adaptive Mix (AdaMix) algorithm for generating perturbed images and a semi-supervised learning paradigm for providing pseudo labels and conducting consistency regularization. AdaMix can be seamlessly applied to the self-training, mean-teacher, and co-training paradigms, resulting in AdaMix-ST, AdaMix-MT, and AdaMix-CT frameworks, respectively.
  • Figure 3: Schematic diagram of the proposed Adaptive Mix algorithm (AdaMix). It performs image mix-up perturbation in a self-paced manner, synthesizing perturbed samples from easy to hard during training based on the model's learning state.
  • Figure 4: Qualitative examples on the ACDC, ISIC, and LA datasets. The yellow dash circles highlight some segmentation regions. Sup: the supervised baseline, MT: Mean-Teacher, CPS: Cross Pseudo Supervision, and GT: Ground Truth.
  • Figure 5: Investigation of the effect of patch size $S$ and maximum No. patches $K$ for AdaMix on the ACDC val set with 10% labeled data.
  • ...and 4 more figures

Theorems & Definitions (1)

  • Theorem 1