Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AMGFormer: Adaptive Multi-Granular Transformer for Brain Tumor Segmentation with Missing Modalities

Chengxiang Guo, Jian Wang, Junhua Fei, Xiao Li, Chunling Chen, Yun Jin

TL;DR

The paper addresses the clinical hurdle of brain tumor segmentation when MRI modalities are missing, which causes large performance variance and undermines trust in automated systems. It introduces AMGFormer, an adaptive multi-granular transformer with three modules: QuadIntegrator Bridge for spatially adaptive fusion, MGAO for efficient sparse attention focused on tumor regions, and MQAE for quality-aware cross-modal enhancement. Across BraTS 2018/2020/2021, AMGFormer achieves state-of-the-art Dice scores while dramatically reducing modality-variance (ET variance <0.5% across 15 modality combinations) and sustaining rapid inference (~1.2 s per volume), demonstrating strong potential for clinical deployment. The method generalizes across datasets, maintains robust single-modality performance, and addresses key stability challenges, suggesting practical impact in settings with incomplete imaging protocols or urgent decision-making.

Abstract

Multimodal MRI is essential for brain tumor segmentation, yet missing modalities in clinical practice cause existing methods to exhibit >40% performance variance across modality combinations, rendering them clinically unreliable. We propose AMGFormer, achieving significantly improved stability through three synergistic modules: (1) QuadIntegrator Bridge (QIB) enabling spatially adaptive fusion maintaining consistent predictions regardless of available modalities, (2) Multi-Granular Attention Orchestrator (MGAO) focusing on pathological regions to reduce background sensitivity, and (3) Modality Quality-Aware Enhancement (MQAE) preventing error propagation from corrupted sequences. On BraTS 2018, our method achieves 89.33% WT, 82.70% TC, 67.23% ET Dice scores with <0.5% variance across 15 modality combinations, solving the stability crisis. Single-modality ET segmentation shows 40-81% relative improvements over state-of-the-art methods. The method generalizes to BraTS 2020/2021, achieving up to 92.44% WT, 89.91% TC, 84.57% ET. The model demonstrates potential for clinical deployment with 1.2s inference. Code: https://github.com/guochengxiangives/AMGFormer.

AMGFormer: Adaptive Multi-Granular Transformer for Brain Tumor Segmentation with Missing Modalities

TL;DR

The paper addresses the clinical hurdle of brain tumor segmentation when MRI modalities are missing, which causes large performance variance and undermines trust in automated systems. It introduces AMGFormer, an adaptive multi-granular transformer with three modules: QuadIntegrator Bridge for spatially adaptive fusion, MGAO for efficient sparse attention focused on tumor regions, and MQAE for quality-aware cross-modal enhancement. Across BraTS 2018/2020/2021, AMGFormer achieves state-of-the-art Dice scores while dramatically reducing modality-variance (ET variance <0.5% across 15 modality combinations) and sustaining rapid inference (~1.2 s per volume), demonstrating strong potential for clinical deployment. The method generalizes across datasets, maintains robust single-modality performance, and addresses key stability challenges, suggesting practical impact in settings with incomplete imaging protocols or urgent decision-making.

Abstract

Multimodal MRI is essential for brain tumor segmentation, yet missing modalities in clinical practice cause existing methods to exhibit >40% performance variance across modality combinations, rendering them clinically unreliable. We propose AMGFormer, achieving significantly improved stability through three synergistic modules: (1) QuadIntegrator Bridge (QIB) enabling spatially adaptive fusion maintaining consistent predictions regardless of available modalities, (2) Multi-Granular Attention Orchestrator (MGAO) focusing on pathological regions to reduce background sensitivity, and (3) Modality Quality-Aware Enhancement (MQAE) preventing error propagation from corrupted sequences. On BraTS 2018, our method achieves 89.33% WT, 82.70% TC, 67.23% ET Dice scores with <0.5% variance across 15 modality combinations, solving the stability crisis. Single-modality ET segmentation shows 40-81% relative improvements over state-of-the-art methods. The method generalizes to BraTS 2020/2021, achieving up to 92.44% WT, 89.91% TC, 84.57% ET. The model demonstrates potential for clinical deployment with 1.2s inference. Code: https://github.com/guochengxiangives/AMGFormer.
Paper Structure (11 sections, 1 equation, 3 figures, 2 tables)

This paper contains 11 sections, 1 equation, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Overview
  4. QuadIntegrator Bridge (QIB)
  5. Multi-Granular Attention Orchestrator (MGAO)
  6. Modality Quality-Aware Enhancement (MQAE)
  7. Experiment
  8. Experimental Setup
  9. Results and Analysis
  10. Ablation Study
  11. Conclusion

Figures (3)

  • Figure 1: AMGFormer architecture. Four MRI modalities processed through encoders (Enc.) with dual-stage MGAO (intra- then inter-modal attention, AG: Adaptive Aggregation). MQAE provides enhancement (Enh.) and quality scores. QIB enables adaptive fusion in decoder (Dec.). AMS-Dec.: auxiliary modality-specific decoders.
  • Figure 2: QIB architecture showing feature alignment, synergistic integration (SIM), and spatially-adaptive fusion controller (SAFC) generating location-specific weights for robust modality fusion.
  • Figure 3: Qualitative comparison across modality availability. From left: input, IAIDNet, DRNet, ours, ground truth. Single-modality row shows severe tumor underestimation in SOTA methods; dual/triple combinations reveal boundary inconsistencies; our method maintains accurate delineation throughout.