Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DiM: $f$-Divergence Minimization Guided Sharpness-Aware Optimization for Semi-supervised Medical Image Segmentation

Bingli Wang, Houcheng Su, Nan Yin, Mengzhu Wang, Li Shen

TL;DR

A sharpness-aware optimization method based on $f-divergence minimization (DiM) for semi-supervised medical image segmentation that enhances the model's stability by fine-tuning the sensitivity of model parameters and improves the model's adaptability to different datasets through the introduction of $f-divergence.

Abstract

As a technique to alleviate the pressure of data annotation, semi-supervised learning (SSL) has attracted widespread attention. In the specific domain of medical image segmentation, semi-supervised methods (SSMIS) have become a research hotspot due to their ability to reduce the need for large amounts of precisely annotated data. SSMIS focuses on enhancing the model's generalization performance by leveraging a small number of labeled samples and a large number of unlabeled samples. The latest sharpness-aware optimization (SAM) technique, which optimizes the model by reducing the sharpness of the loss function, has shown significant success in SSMIS. However, SAM and its variants may not fully account for the distribution differences between different datasets. To address this issue, we propose a sharpness-aware optimization method based on $f$-divergence minimization (DiM) for semi-supervised medical image segmentation. This method enhances the model's stability by fine-tuning the sensitivity of model parameters and improves the model's adaptability to different datasets through the introduction of $f$-divergence. By reducing $f$-divergence, the DiM method not only improves the performance balance between the source and target datasets but also prevents performance degradation due to overfitting on the source dataset.

DiM: $f$-Divergence Minimization Guided Sharpness-Aware Optimization for Semi-supervised Medical Image Segmentation

TL;DR

A sharpness-aware optimization method based on f-divergence.

Abstract

As a technique to alleviate the pressure of data annotation, semi-supervised learning (SSL) has attracted widespread attention. In the specific domain of medical image segmentation, semi-supervised methods (SSMIS) have become a research hotspot due to their ability to reduce the need for large amounts of precisely annotated data. SSMIS focuses on enhancing the model's generalization performance by leveraging a small number of labeled samples and a large number of unlabeled samples. The latest sharpness-aware optimization (SAM) technique, which optimizes the model by reducing the sharpness of the loss function, has shown significant success in SSMIS. However, SAM and its variants may not fully account for the distribution differences between different datasets. To address this issue, we propose a sharpness-aware optimization method based on -divergence minimization (DiM) for semi-supervised medical image segmentation. This method enhances the model's stability by fine-tuning the sensitivity of model parameters and improves the model's adaptability to different datasets through the introduction of -divergence. By reducing -divergence, the DiM method not only improves the performance balance between the source and target datasets but also prevents performance degradation due to overfitting on the source dataset.

Paper Structure

This paper contains 16 sections, 11 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Semi-supervised Medical Image Segmentation
  4. Unsupervised Domain Adaptation
  5. Sharpness-Aware Minimization (SAM)
  6. Methods
  7. Aligning Features via $f$-Divergence
  8. Sharpness-Aware Entropy Minimization
  9. Loss Function
  10. Experiments
  11. Experiment Datasets
  12. Comparison Methods and Settings
  13. Comparison with State-of-the-Art Methods
  14. Ablation Study
  15. Visualization Analysis
  16. ...and 1 more sections

Figures (5)

  • Figure 1: A T-sne visualization analysis was performed on the Fundus dataset experiment.
  • Figure 2: Visual comparison of segmentation results on Fundus dataset across different models. Red and green represent the Optical Cup and Disc, respectively. The first row shows segmentation results on test samples from the labeled domain (Domain 1), while the second row presents results on test samples from a different domain (Domain 4).
  • Figure 3: Visual results on Prostate dataset.
  • Figure 4: Validation Loss and Dice across four domains on Fundus dataset.
  • Figure 5: Validation Loss and Dice for Domain 1 on the Prostate dataset.