Unveiling Incomplete Modality Brain Tumor Segmentation: Leveraging Masked Predicted Auto-Encoder and Divergence Learning
Zhongao Sun, Jiameng Li, Yuhan Wang, Jiarong Cheng, Qing Zhou, Chun Li
TL;DR
The work tackles incomplete multimodal brain tumor segmentation in multi-modal MRI by introducing a masked predicted auto-encoder pre-training scheme and Hölder-divergence-based knowledge distillation, built on a 3D Swin Transformer backbone. It enables learning robust representations from incomplete data and effectively transfers knowledge from full-modality teachers to missing-modality students via a $D_alpha^H(p:q)$-based loss. Empirical results on BraTS2018 and BraTS2020 demonstrate state-of-the-art performance across missing modality scenarios, with particular strength when multiple modalities are absent, albeit with increased training complexity and sensitivity to hyperparameters. This approach offers a practical, robust direction for clinical pipelines where data completeness cannot be guaranteed."
Abstract
Brain tumor segmentation remains a significant challenge, particularly in the context of multi-modal magnetic resonance imaging (MRI) where missing modality images are common in clinical settings, leading to reduced segmentation accuracy. To address this issue, we propose a novel strategy, which is called masked predicted pre-training, enabling robust feature learning from incomplete modality data. Additionally, in the fine-tuning phase, we utilize a knowledge distillation technique to align features between complete and missing modality data, simultaneously enhancing model robustness. Notably, we leverage the Holder pseudo-divergence instead of the KLD for distillation loss, offering improve mathematical interpretability and properties. Extensive experiments on the BRATS2018 and BRATS2020 datasets demonstrate significant performance enhancements compared to existing state-of-the-art methods.