Deep Mutual Learning among Partially Labeled Datasets for Multi-Organ Segmentation

TL;DR

This work tackles the challenge of missing full annotations in multi-organ segmentation by leveraging numerous partially labeled datasets. It introduces a two-stage mutual learning framework: stage one trains Partial-Organ Segmentation models with Difference Mutual Learning (Prediction Difference and Feature Difference) to produce high-quality pseudo labels; stage two trains Full-Organ Segmentation models with Similarity Mutual Learning (Prediction Similarity and Dynamic Feature Similarity) using combined and true cross-dataset labels and dynamic feature transfer. Extensive experiments across head/neck, chest, abdomen, and pelvis demonstrate state-of-the-art performance and robust ablations validating each component. The approach effectively exploits cross-dataset supervision and has practical potential to enable accurate, scalable multi-organ segmentation in clinical settings when fully labeled data are scarce.

Abstract

The task of labeling multiple organs for segmentation is a complex and time-consuming process, resulting in a scarcity of comprehensively labeled multi-organ datasets while the emergence of numerous partially labeled datasets. Current methods are inadequate in effectively utilizing the supervised information available from these datasets, thereby impeding the progress in improving the segmentation accuracy. This paper proposes a two-stage multi-organ segmentation method based on mutual learning, aiming to improve multi-organ segmentation performance by complementing information among partially labeled datasets. In the first stage, each partial-organ segmentation model utilizes the non-overlapping organ labels from different datasets and the distinct organ features extracted by different models, introducing additional mutual difference learning to generate higher quality pseudo labels for unlabeled organs. In the second stage, each full-organ segmentation model is supervised by fully labeled datasets with pseudo labels and leverages true labels from other datasets, while dynamically sharing accurate features across different models, introducing additional mutual similarity learning to enhance multi-organ segmentation performance. Extensive experiments were conducted on nine datasets that included the head and neck, chest, abdomen, and pelvis. The results indicate that our method has achieved SOTA performance in segmentation tasks that rely on partial labels, and the ablation studies have thoroughly confirmed the efficacy of the mutual learning mechanism.

Figures (9)

• Figure 1: Four types of methods to address partially labeled datasets for training multi-organ segmentation models. (a) multiple networks. (b) pseudo-labeling. (c) channel adjustment. (d) conditional information guidance.
• Figure 2: Main idea of our method. (a) each model in stage 1 is supervised by three kinds of information, including the labels in current dataset, the labels of the other datasets, and the features of other organs extracted by other models. (b) models in stage 2 are also supervised by three kinds of information, including the combined labels (with both the true labels and pseudo labels), the true labels of the other datasets, and the correct features extracted by the other models.
• Figure 3: The overall framework of our method. The first stage involves training multiple models to segment partial organs (from left to right). In addition to employing its own labels for supervised learning, each model participates in additional mutual Prediction Difference (PD) and Feature Difference (FD) learning. Then the trained models generate pseudo labels for other datasets, resulting in a combined labeled dataset. In the second stage, multiple models capable of segmenting all organs are trained (from right to left). During training, each model is supervised not only by the combined labels but also participates in additional mutual Prediction Similarity (PS) and Dynamic Feature Similarity (DFS) learning.
• Figure 4: Visualization of segmentation results for each method in head and neck. The red dashed box represents the selected and enlarged region, while the yellow dashed box only represents the selected region.
• Figure 5: Visualization of segmentation results for each method in chest. The red dashed box represents the selected and enlarged region, while the yellow dashed box only represents the selected region.