Bi-CRCL: Bidirectional Conservative-Radical Complementary Learning with Pre-trained Foundation Models for Class-incremental Medical Image Analysis

Abstract

Class-incremental learning (CIL) in medical image-guided diagnosis requires retaining prior diagnostic knowledge while adapting to newly emerging disease categories, which is critical for scalable clinical deployment. This problem is particularly challenging due to heterogeneous data and privacy constraints that prevent memory replay. Although pretrained foundation models (PFMs) have advanced general-domain CIL, their potential in medical imaging remains underexplored, where domain-specific adaptation is essential yet difficult due to anatomical complexity and inter-institutional heterogeneity. To address this gap, we conduct a systematic benchmark of recent PFM-based CIL methods and propose Bidirectional Conservative-Radical Complementary Learning (Bi-CRCL), a dual-learner framework inspired by complementary learning systems. Bi-CRCL integrates a conservative learner that preserves prior knowledge through stability-oriented updates and a radical learner that rapidly adapts to new categories via plasticity-oriented learning. A bidirectional interaction mechanism enables forward transfer and backward consolidation, allowing continual integration of new knowledge while mitigating catastrophic forgetting. During inference, outputs from both learners are adaptively fused for robust predictions. Experiments on five medical imaging datasets demonstrate consistent improvements over state-of-the-art methods under diverse settings, including cross-dataset shifts and varying task configurations.

Figures (4)

• Figure 1: Illustration of conventional vs. PFM-based medical CIL. Conventional CIL trains models from scratch, similar to teaching an infant to learn from zero experience. PFM-based CIL builds on PFMs, analogous to guiding an experienced adult to adapt efficiently to new tasks. Both paradigms share the fundamental challenge of catastrophic forgetting when learning new classes sequentially.
• Figure 2: Overview of our Bi-CRCL framework. (a) Initialized Domain Alignment: At task $t=1$, the general-domain PFM adapts to the medical domain via adapter tuning to enable efficient PFM transfer while preserving generalization; (b) Continual Bidirectional Complementary Learning: For $t>1$, a radical learner incrementally learns new classes and exchanges knowledge with the conservative learner through bidirectional consolidation and re-initialization, emulating hippocampus–neocortex coordination; (c) Collaborative Inference: During inference, outputs from both learners are projected via analytical classifiers and adaptively fused to produce robust task-agnostic predictions.
• Figure 3: An illustration of the diverse medical imaging tasks addressed by our model, spanning lesion classification (dermoscopic images), organ classification (CT scans), blood cell classification (microscopy), tissue classification (histopathology), and disease detection (CT & X-ray scans), with representative example classes from each dataset.
• Figure 4: The performance curves across learning sessions on five medical datasets. Bi-CRCL consistently achieves the best overall performance and exhibits minimal degradation as class number increases.