Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Modality-Agnostic Continual Domain-Incremental Brain Lesion Segmentation

Yousef Sadegheih, Dorit Merhof, Pratibha Kumari

TL;DR

This work tackles domain-incremental brain lesion segmentation under heterogeneous MRI modalities by proposing CLMU-Net, a replay-based continual learning framework. It fuses modality-flexible input through channel inflation and random modality drop, domain-conditioned textual guidance via cross-attention with BioBERT-derived prompts, and a lesion-aware replay buffer that prioritizes representative and difficult samples. Across five diverse 3D brain MRI datasets, CLMU-Net demonstrates robust performance with limited memory, outperforming both buffer-free and buffer-based baselines and showing significant improvements under evolving modality conditions. The combination of flexible modality handling, targeted replay, and global contextual cues offers a practical path toward clinically deployable continual medical image segmentation.

Abstract

Brain lesion segmentation from multi-modal MRI often assumes fixed modality sets or predefined pathologies, making existing models difficult to adapt across cohorts and imaging protocols. Continual learning (CL) offers a natural solution but current approaches either impose a maximum modality configuration or suffer from severe forgetting in buffer-free settings. We introduce CLMU-Net, a replay-based CL framework for 3D brain lesion segmentation that supports arbitrary and variable modality combinations without requiring prior knowledge of the maximum set. A conceptually simple yet effective channel-inflation strategy maps any modality subset into a unified multi-channel representation, enabling a single model to operate across diverse datasets. To enrich inherently local 3D patch features, we incorporate lightweight domain-conditioned textual embeddings that provide global modality-disease context for each training case. Forgetting is further reduced through principled replay using a compact buffer composed of both prototypical and challenging samples. Experiments on five heterogeneous MRI brain datasets demonstrate that CLMU-Net consistently outperforms popular CL baselines. Notably, our method yields an average Dice score improvement of $\geq$ 18\% while remaining robust under heterogeneous-modality conditions. These findings underscore the value of flexible modality handling, targeted replay, and global contextual cues for continual medical image segmentation. Our implementation is available at https://github.com/xmindflow/CLMU-Net.

Towards Modality-Agnostic Continual Domain-Incremental Brain Lesion Segmentation

TL;DR

This work tackles domain-incremental brain lesion segmentation under heterogeneous MRI modalities by proposing CLMU-Net, a replay-based continual learning framework. It fuses modality-flexible input through channel inflation and random modality drop, domain-conditioned textual guidance via cross-attention with BioBERT-derived prompts, and a lesion-aware replay buffer that prioritizes representative and difficult samples. Across five diverse 3D brain MRI datasets, CLMU-Net demonstrates robust performance with limited memory, outperforming both buffer-free and buffer-based baselines and showing significant improvements under evolving modality conditions. The combination of flexible modality handling, targeted replay, and global contextual cues offers a practical path toward clinically deployable continual medical image segmentation.

Abstract

Brain lesion segmentation from multi-modal MRI often assumes fixed modality sets or predefined pathologies, making existing models difficult to adapt across cohorts and imaging protocols. Continual learning (CL) offers a natural solution but current approaches either impose a maximum modality configuration or suffer from severe forgetting in buffer-free settings. We introduce CLMU-Net, a replay-based CL framework for 3D brain lesion segmentation that supports arbitrary and variable modality combinations without requiring prior knowledge of the maximum set. A conceptually simple yet effective channel-inflation strategy maps any modality subset into a unified multi-channel representation, enabling a single model to operate across diverse datasets. To enrich inherently local 3D patch features, we incorporate lightweight domain-conditioned textual embeddings that provide global modality-disease context for each training case. Forgetting is further reduced through principled replay using a compact buffer composed of both prototypical and challenging samples. Experiments on five heterogeneous MRI brain datasets demonstrate that CLMU-Net consistently outperforms popular CL baselines. Notably, our method yields an average Dice score improvement of 18\% while remaining robust under heterogeneous-modality conditions. These findings underscore the value of flexible modality handling, targeted replay, and global contextual cues for continual medical image segmentation. Our implementation is available at https://github.com/xmindflow/CLMU-Net.
Paper Structure (14 sections, 3 figures, 3 tables)

This paper contains 14 sections, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Buffer Selection Criteria
  4. Representative samples:
  5. Difficult samples:
  6. Final Buffer Composition and Management
  7. Domain-conditioned Textual Guidance
  8. Modality-flexible Segmentation
  9. Experiments
  10. Datasets, Experimental Setup, Evaluation Metrics, CL Benchmarks
  11. Results
  12. Ablation
  13. Limitations and Future Directions
  14. Conclusion

Figures (3)

  • Figure 1: Overview of the CLMU-Net framework.
  • Figure 2: Modality-flexible design: varying episode-wise modalities (top), channel inflation for new modalities (middle), and RMD for modality-agnostic training (bottom).
  • Figure 3: ER (dashed) vs. CLMU-Net (solid) across $\beta$ in $S1$, $S2$ (left/right: AVG/ILM).