EndoCIL: A Class-Incremental Learning Framework for Endoscopic Image Classification

TL;DR

EndoCIL tackles the crucial problem of class-incremental learning for endoscopic image classification by addressing domain shift and dual-phase class imbalance. It introduces three components: MDBR for distribution-aligned exemplar replay, PRCBL to balance learning across old and new classes, and CFG to mitigate classifier bias toward newly added classes. The method achieves consistent improvements over state-of-the-art CIL approaches across four public endoscopic datasets and various buffer sizes, validating its effectiveness and robustness. The work offers a practical, scalable approach for lifelong diagnostic systems in clinical endoscopy, combining theoretical grounding with extensive empirical validation.

Abstract

Class-incremental learning (CIL) for endoscopic image analysis is crucial for real-world clinical applications, where diagnostic models should continuously adapt to evolving clinical data while retaining performance on previously learned ones. However, existing replay-based CIL methods fail to effectively mitigate catastrophic forgetting due to severe domain discrepancies and class imbalance inherent in endoscopic imaging. To tackle these challenges, we propose EndoCIL, a novel and unified CIL framework specifically tailored for endoscopic image diagnosis. EndoCIL incorporates three key components: Maximum Mean Discrepancy Based Replay (MDBR), employing a distribution-aligned greedy strategy to select diverse and representative exemplars, Prior Regularized Class Balanced Loss (PRCBL), designed to alleviate both inter-phase and intra-phase class imbalance by integrating prior class distributions and balance weights into the loss function, and Calibration of Fully-Connected Gradients (CFG), which adjusts the classifier gradients to mitigate bias toward new classes. Extensive experiments conducted on four public endoscopic datasets demonstrate that EndoCIL generally outperforms state-of-the-art CIL methods across varying buffer sizes and evaluation metrics. The proposed framework effectively balances stability and plasticity in lifelong endoscopic diagnosis, showing promising potential for clinical scalability and deployment.

Figures (8)

• Figure 1: Illustration of key challenges: (A) shows the limitations of the mean exemplar selection strategy, addressed via MMD-Based Replay to preserve feature diversity; (B) highlights the dual-phase class imbalance, mitigated by Prior Regularized Class Balanced Loss and Calibration of Fully-Connected Gradients.
• Figure 2: Overview of the proposed method. When task $t = 0$, the model is trained from scratch. For each subsequent task ($t > 0$), a representative subset is selected via the MDBR (B) to approximate the feature distribution of previous tasks. Then, the data from the current task and the stored exemplars in the memory buffer are jointly used as training input. During training, the logits from the previous model are aligned with those of the current model using a distillation loss. The prediction loss is computed using the PRCBL (D). The gradients induced by both losses on the fully connected (FC) layer are modulated by the CFG (C), mitigating the classifier's bias toward new classes.
• Figure 3: Class samples distribution of different datasets.
• Figure 4: Performances of different methods across learning sessions on four endoscopic datasets.
• Figure 5: UMAP-based visualization of feature distributions for MMD-based selection strategy across four endoscopic datasets. Light circles ($\circ$) denote all training samples and triangles ($\blacktriangle$) indicate selections by MMD-based method.