Cross-model Mutual Learning for Exemplar-based Medical Image Segmentation

TL;DR

This work tackles exemplar-based medical image segmentation where only a single annotated exemplar is available. It introduces CMEMS, a Cross-model Mutual Learning framework using two identically structured segmentation models that mutually exploit unlabeled data at image and feature levels through cross-model perturbations and multi-level feature perturbations. The method integrates exemplar and synthetic data with dynamic pseudo-labels to foster prediction and feature consistency across models, mitigating confirmation bias. Empirical results on Synapse and ACDC demonstrate state-of-the-art performance under extremely limited supervision, with qualitative evidence of robust and accurate segmentations in challenging medical images.

Abstract

Medical image segmentation typically demands extensive dense annotations for model training, which is both time-consuming and skill-intensive. To mitigate this burden, exemplar-based medical image segmentation methods have been introduced to achieve effective training with only one annotated image. In this paper, we introduce a novel Cross-model Mutual learning framework for Exemplar-based Medical image Segmentation (CMEMS), which leverages two models to mutually excavate implicit information from unlabeled data at multiple granularities. CMEMS can eliminate confirmation bias and enable collaborative training to learn complementary information by enforcing consistency at different granularities across models. Concretely, cross-model image perturbation based mutual learning is devised by using weakly perturbed images to generate high-confidence pseudo-labels, supervising predictions of strongly perturbed images across models. This approach enables joint pursuit of prediction consistency at the image granularity. Moreover, cross-model multi-level feature perturbation based mutual learning is designed by letting pseudo-labels supervise predictions from perturbed multi-level features with different resolutions, which can broaden the perturbation space and enhance the robustness of our framework. CMEMS is jointly trained using exemplar data, synthetic data, and unlabeled data in an end-to-end manner. Experimental results on two medical image datasets indicate that the proposed CMEMS outperforms the state-of-the-art segmentation methods with extremely limited supervision.

Figures (6)

• Figure 1: Illustration of the proposed idea. The proposed CMEMS leverages two mutual learning models to excavate implicit information from unlabeled data for exemplar-based medical image segmentation.
• Figure 2: An overview of the proposed CMEMS. Firstly, a synthetic dataset $\mathcal{D}_{S}$ is generated from an exemplar dataset $\mathcal{D}_{E}$ using the $\Omega$ method. Both datasets are then fed into segmentation networks $\mathcal{N}_{m} (m \in \{1,2\})$ to calculate $L_{e}$ and $L_{s}$. Next, the unlabeled image $I_{u}$ is fed into two segmentation networks using weak and strong perturbations, respectively, to calculate $L_{cmip}$ by cross-model image perturbation based mutual learning and to calculate $L_{cmfp}$ via cross-model multi-level feature perturbation based mutual learning. Finally, the two segmentation networks are optimized collaboratively by computing all loss functions.
• Figure 3: Impact of the weight (a) $\lambda_{cmip}$ and (b) $\lambda_{cmfp}$ on the performance on the ACDC dataset.
• Figure 4: Impact of (a) the confidence threshold $\tau$, and (b) the intensity factor $\alpha$ on the performance on the ACDC dataset.
• Figure 5: Visual examples of the segmentation results obtained by the proposed CMEMS framework and other state-of-the-art methods on the Synapse and ACDC datasets. The first two columns display the input images and the corresponding ground truth labels. The last column shows visualization of the segmentation results generated by CMEMS. The remaining columns show the results obtained by other methods.