Mitigating Catastrophic Forgetting in the Incremental Learning of Medical Images

TL;DR

This work tackles catastrophic forgetting in medical-image incremental learning by integrating knowledge distillation with a lightweight past-data generator. A fixed Teacher guides a Student through a KD loss that combines feature-attention matching and covariance regularization, while a shallow VAE synthesizes past-task samples to support replay without storing real data. Experiments on PI-CAI, OCT, PathMNIST, and CIFAR-10 show that the proposed KD-based IL approach maintains prior knowledge and achieves performance close to or exceeding baselines, with ablations demonstrating the necessity of both KD components. The method offers privacy-preserving, data-efficient continual learning suitable for multi-center medical imaging analysis and could be extended to additional modalities and tasks.

Abstract

This paper proposes an Incremental Learning (IL) approach to enhance the accuracy and efficiency of deep learning models in analyzing T2-weighted (T2w) MRI medical images prostate cancer detection using the PI-CAI dataset. We used multiple health centers' artificial intelligence and radiology data, focused on different tasks that looked at prostate cancer detection using MRI (PI-CAI). We utilized Knowledge Distillation (KD), as it employs generated images from past tasks to guide the training of models for subsequent tasks. The approach yielded improved performance and faster convergence of the models. To demonstrate the versatility and robustness of our approach, we evaluated it on the PI-CAI dataset, a diverse set of medical imaging modalities including OCT and PathMNIST, and the benchmark continual learning dataset CIFAR-10. Our results indicate that KD can be a promising technique for IL in medical image analysis in which data is sourced from individual health centers and the storage of large datasets is not feasible. By using generated images from prior tasks, our method enables the model to retain and apply previously acquired knowledge without direct access to the original data.

Figures (3)

• Figure 1: a) Separable features. b) Discriminative features.
• Figure 2: Overview diagram of the proposed framework: The triplet loss $\mathcal{L}_{\text{tri}}(x_k)$ is used to train the Student model $M_k$ on the current task using real data ($x_k \in D_k$). Distillation (KD) loss, defined as $\mathcal{L}_{\text{KD}} = \mathcal{L}_{\text{FAM}} + \mathcal{L}_{\text{cov}}$, mitigates catastrophic forgetting. A light Variational Autoencoder (VAE) serves as a generator, with the loss ($\mathcal{L}_{G}$) maximizing the distance between the Student ($M_k$) and Teacher ($M_{k-1}$) adversarially, ensuring synthetic images resemble previous task data distribution and enabling effective knowledge transfer.
• Figure 3: $A_k$ (%) for the ablation study conducted on two datasets.