Class Incremental Medical Image Segmentation via Prototype-Guided Calibration and Dual-Aligned Distillation

TL;DR

This work tackles the problem of class incremental semantic segmentation in 3D medical images, where previously learned structures must be preserved while new anatomy is learned without old labels. It introduces Prototype-Guided Calibration Distillation (PGCD) to adapt distillation strength across spatial regions and channels using class prototypes, and Dual-Aligned Prototype Distillation (DAPD) to align old-class representations with both global and local prototypes, including a dynamically updated background prototype. The proposed approach yields state-of-the-art results on two public multi-organ benchmarks (BTCV and WORD), showing strong retention of old classes and effective learning of new ones across various incremental schedules. The findings demonstrate robustness and generalization, with implications for reliable, privacy-aware continual learning in clinical segmentation tasks.

Abstract

Class incremental medical image segmentation (CIMIS) aims to preserve knowledge of previously learned classes while learning new ones without relying on old-class labels. However, existing methods 1) either adopt one-size-fits-all strategies that treat all spatial regions and feature channels equally, which may hinder the preservation of accurate old knowledge, 2) or focus solely on aligning local prototypes with global ones for old classes while overlooking their local representations in new data, leading to knowledge degradation. To mitigate the above issues, we propose Prototype-Guided Calibration Distillation (PGCD) and Dual-Aligned Prototype Distillation (DAPD) for CIMIS in this paper. Specifically, PGCD exploits prototype-to-feature similarity to calibrate class-specific distillation intensity in different spatial regions, effectively reinforcing reliable old knowledge and suppressing misleading information from old classes. Complementarily, DAPD aligns the local prototypes of old classes extracted from the current model with both global prototypes and local prototypes, further enhancing segmentation performance on old categories. Comprehensive evaluations on two widely used multi-organ segmentation benchmarks demonstrate that our method outperforms state-of-the-art methods, highlighting its robustness and generalization capabilities.

Figures (4)

• Figure 1: The conceptual comparison of (a) previous one-size-fits-all knowledge distillation and our (b) prototype-guided calibrated distillation. Unlike previous approaches that treated all regions and channels equally, our method calibrates the distillation process under the guidance of class prototypes, thereby better preserving useful old knowledge while suppressing misleading signals. The proposed DAPD further improves the segmentation performance of previously learned classes.
• Figure 2: Overview of our proposed method. At step $t$, the model $f_{\theta}^{t}$ learns a new class (left kidney) while retaining knowledge of three old classes (liver, right kidney, and spleen). As shown in the top-left subfigure, global prototypes $\left \{ \mathbf{p}_c \mid c \in \mathcal{C}^{1:t-1} \cup 0 \right \}$ for the old classes are extracted and stored after step $t-1$. The bottom-left subfigure illustrates that, at the current step $t$, both the frozen model $f_{\theta}^{t-1}$ and the current model $f_{\theta}^{t}$ extract local prototypes $\left \{ \mathbf{\hat{p}}_c^{t-1} \mid c \in \mathcal{C}^{1:t-1} \cup 0 \right \}$ and $\left \{ \mathbf{\hat{p}}_c^{t} \mid c \in \mathcal{C}^{1:t} \cup 0 \right \}$, respectively. DAPD simultaneously aligns $\mathbf{\hat{p}}_c^{t}$ with both $\mathbf{\hat{p}}_c^{t-1}$ and $\mathbf{{p}}_c$ to enhance knowledge distillation for old classes. PGCD calibrates the distillation process across spatial regions and feature channels under the guidance of class prototypes.
• Figure 3: Qualitative comparison of segmentation results between the proposed method and SOTA approaches on BTCV 4-4.
• Figure 4: The average DSC (%) of different methods at each incremental step on the BTCV 2-2 setting.