Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ModelMix: A New Model-Mixup Strategy to Minimize Vicinal Risk across Tasks for Few-scribble based Cardiac Segmentation

Ke Zhang, Vishal M. Patel

TL;DR

The paper tackles the annotation bottleneck in medical image segmentation by leveraging few-scribble labels and exploiting cross-task correlations. It introduces ModelMix, a model-mixup strategy that linearly interpolates encoder parameters from task-specific models to form virtual mixed models, guided by vicinal risk minimization. Two vicinal regularization terms are proposed: an unsupervised consistency term and a scribble-supervised term, plus an invariant loss on image-level augmentations. On three cardiac MRI datasets (ACDC, MSCMRseg, MyoPS), ModelMix achieves state-of-the-art performance under few-scribble supervision, demonstrating effective cross-task priors and robustness.

Abstract

Pixel-level dense labeling is both resource-intensive and time-consuming, whereas weak labels such as scribble present a more feasible alternative to full annotations. However, training segmentation networks with weak supervision from scribbles remains challenging. Inspired by the fact that different segmentation tasks can be correlated with each other, we introduce a new approach to few-scribble supervised segmentation based on model parameter interpolation, termed as ModelMix. Leveraging the prior knowledge that linearly interpolating convolution kernels and bias terms should result in linear interpolations of the corresponding feature vectors, ModelMix constructs virtual models using convex combinations of convolutional parameters from separate encoders. We then regularize the model set to minimize vicinal risk across tasks in both unsupervised and scribble-supervised way. Validated on three open datasets, i.e., ACDC, MSCMRseg, and MyoPS, our few-scribble guided ModelMix significantly surpasses the performance of the state-of-the-art scribble supervised methods.

ModelMix: A New Model-Mixup Strategy to Minimize Vicinal Risk across Tasks for Few-scribble based Cardiac Segmentation

TL;DR

The paper tackles the annotation bottleneck in medical image segmentation by leveraging few-scribble labels and exploiting cross-task correlations. It introduces ModelMix, a model-mixup strategy that linearly interpolates encoder parameters from task-specific models to form virtual mixed models, guided by vicinal risk minimization. Two vicinal regularization terms are proposed: an unsupervised consistency term and a scribble-supervised term, plus an invariant loss on image-level augmentations. On three cardiac MRI datasets (ACDC, MSCMRseg, MyoPS), ModelMix achieves state-of-the-art performance under few-scribble supervision, demonstrating effective cross-task priors and robustness.

Abstract

Pixel-level dense labeling is both resource-intensive and time-consuming, whereas weak labels such as scribble present a more feasible alternative to full annotations. However, training segmentation networks with weak supervision from scribbles remains challenging. Inspired by the fact that different segmentation tasks can be correlated with each other, we introduce a new approach to few-scribble supervised segmentation based on model parameter interpolation, termed as ModelMix. Leveraging the prior knowledge that linearly interpolating convolution kernels and bias terms should result in linear interpolations of the corresponding feature vectors, ModelMix constructs virtual models using convex combinations of convolutional parameters from separate encoders. We then regularize the model set to minimize vicinal risk across tasks in both unsupervised and scribble-supervised way. Validated on three open datasets, i.e., ACDC, MSCMRseg, and MyoPS, our few-scribble guided ModelMix significantly surpasses the performance of the state-of-the-art scribble supervised methods.
Paper Structure (10 sections, 8 equations, 3 figures, 3 tables)

This paper contains 10 sections, 8 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Train for separate tasks
  4. Minimize vicinal risk across tasks
  5. Vicinal regularization
  6. Experiment
  7. Experiment Setup
  8. Ablation study
  9. Performance and Comparisons
  10. Conclusion

Figures (3)

  • Figure 1: Overview of the proposed ModelMix framework for cardiac segmentation from scribble supervision.
  • Figure 2: The qualitative comparison on MSCMRseg dataset. The three images are the worst, median and best cases selected by the average Dice Score.
  • Figure 3: The qualitative comparison on MyoPS dataset. The two images are the median and best cases selected by the average Dice Score.