SER-Diff: Synthetic Error Replay Diffusion for Incremental Brain Tumor Segmentation

TL;DR

Incremental brain tumor segmentation must adapt to evolving clinical data while avoiding catastrophic forgetting. The authors propose SER-Diff, which unifies diffusion-based segmentation refinement with incremental learning by replaying synthetic error maps $\hat{E}_{old}$ generated by a frozen teacher diffusion model and conditioning refinement on multimodal MRI. A dual-loss objective combines segmentation losses for the current task with a knowledge-distillation term and covariance regularization to preserve past representations. Empirical results on BraTS2020, BraTS2021, and BraTS2023 show state-of-the-art Dice scores (95.8%, 94.9%, 94.6%) and low HD95 (4.4, 4.7, 4.9 mm), indicating improved accuracy and anatomical coherence while mitigating forgetting. This approach reduces the need to store prior data and offers a privacy-friendly, scalable solution for continual medical image segmentation in evolving clinical environments.

Abstract

Incremental brain tumor segmentation is critical for models that must adapt to evolving clinical datasets without retraining on all prior data. However, catastrophic forgetting, where models lose previously acquired knowledge, remains a major obstacle. Recent incremental learning frameworks with knowledge distillation partially mitigate forgetting but rely heavily on generative replay or auxiliary storage. Meanwhile, diffusion models have proven effective for refining tumor segmentations, but have not been explored in incremental learning contexts. We propose Synthetic Error Replay Diffusion (SER-Diff), the first framework that unifies diffusion-based refinement with incremental learning. SER-Diff leverages a frozen teacher diffusion model to generate synthetic error maps from past tasks, which are replayed during training on new tasks. A dual-loss formulation combining Dice loss for new data and knowledge distillation loss for replayed errors ensures both adaptability and retention. Experiments on BraTS2020, BraTS2021, and BraTS2023 demonstrate that SER-Diff consistently outperforms prior methods. It achieves the highest Dice scores of 95.8\%, 94.9\%, and 94.6\%, along with the lowest HD95 values of 4.4 mm, 4.7 mm, and 4.9 mm, respectively. These results indicate that SER-Diff not only mitigates catastrophic forgetting but also delivers more accurate and anatomically coherent segmentations across evolving datasets.