Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CLoE: Expert Consistency Learning for Missing Modality Segmentation

Xinyu Tong, Meihua Zhou, Bowu Fan, Haitao Li

TL;DR

Consistency Learning of Experts (CLoE), a consistency-driven framework for missing-modality segmentation that preserves strong performance when all modalities are available, is proposed and outperforms state-of-the-art methods in incomplete multimodal segmentation.

Abstract

Multimodal medical image segmentation often faces missing modalities at inference, which induces disagreement among modality experts and makes fusion unstable, particularly on small foreground structures. We propose Consistency Learning of Experts (CLoE), a consistency-driven framework for missing-modality segmentation that preserves strong performance when all modalities are available. CLoE formulates robustness as decision-level expert consistency control and introduces a dual-branch Expert Consistency Learning objective. Modality Expert Consistency enforces global agreement among expert predictions to reduce case-wise drift under partial inputs, while Region Expert Consistency emphasizes agreement on clinically critical foreground regions to avoid background-dominated regularization. We further map consistency scores to modality reliability weights using a lightweight gating network, enabling reliability-aware feature recalibration before fusion. Extensive experiments on BraTS 2020 and MSD Prostate demonstrate that CLoE outperforms state-of-the-art methods in incomplete multimodal segmentation, while exhibiting strong cross-dataset generalization and improving robustness on clinically critical structures.

CLoE: Expert Consistency Learning for Missing Modality Segmentation

TL;DR

Consistency Learning of Experts (CLoE), a consistency-driven framework for missing-modality segmentation that preserves strong performance when all modalities are available, is proposed and outperforms state-of-the-art methods in incomplete multimodal segmentation.

Abstract

Multimodal medical image segmentation often faces missing modalities at inference, which induces disagreement among modality experts and makes fusion unstable, particularly on small foreground structures. We propose Consistency Learning of Experts (CLoE), a consistency-driven framework for missing-modality segmentation that preserves strong performance when all modalities are available. CLoE formulates robustness as decision-level expert consistency control and introduces a dual-branch Expert Consistency Learning objective. Modality Expert Consistency enforces global agreement among expert predictions to reduce case-wise drift under partial inputs, while Region Expert Consistency emphasizes agreement on clinically critical foreground regions to avoid background-dominated regularization. We further map consistency scores to modality reliability weights using a lightweight gating network, enabling reliability-aware feature recalibration before fusion. Extensive experiments on BraTS 2020 and MSD Prostate demonstrate that CLoE outperforms state-of-the-art methods in incomplete multimodal segmentation, while exhibiting strong cross-dataset generalization and improving robustness on clinically critical structures.
Paper Structure (8 sections, 11 equations, 2 figures, 3 tables, 1 algorithm)

This paper contains 8 sections, 11 equations, 2 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Method
  3. Overview of Method
  4. Expert Consistency Measurement
  5. Consistency-Driven Dynamic Gating
  6. Learning Process
  7. Experiments and Results
  8. Conclusion

Figures (2)

  • Figure 1: Overview of our proposed CLoE. This framework first extracts modality-specific features Mx from input volumes, undergoes regularization Mpred, and utilizes the ECL module to compute its Modality and Region Expert Consistency scores. Finally, it derives modality reliability weights via a lightweight gating network and performs consistency-driven weighted fusion to achieve robust missing modal segmentation.
  • Figure 2: (a) Visualization of the input modalities. (b) General MedSAM model prediction with bounding box prompt. (c) CLoE predicted segmentation maps.