Continual Learning in Medical Image Analysis: A Comprehensive Review of Recent Advancements and Future Prospects

TL;DR

This systematic review synthesizes recent progress on continual learning (CL) in medical image analysis, emphasizing how data drifts and catastrophic forgetting challenge clinical deployment. It categorizes CL approaches into rehearsal, regularization, architectural, and hybrid strategies, and maps them onto five medical CL scenarios: instance, class, task, domain, and hybrid, plus simulated settings. It discusses supervision levels, evaluation metrics (backward and forward transfer), and practical considerations such as privacy, annotation cost, and interpretability, highlighting the lack of standard benchmarks like in natural image domains. The work also surveys applications across radiology and histopathology, analyzes comparative studies, and outlines open challenges and future directions, including unsupervised CL, scalable benchmarks, and domain-agnostic methods. Overall, the review provides a comprehensive roadmap for advancing CL in dynamic medical imaging environments and improving long-term diagnostic reliability and patient care.

Abstract

Medical imaging analysis has witnessed remarkable advancements even surpassing human-level performance in recent years, driven by the rapid development of advanced deep-learning algorithms. However, when the inference dataset slightly differs from what the model has seen during one-time training, the model performance is greatly compromised. The situation requires restarting the training process using both the old and the new data which is computationally costly, does not align with the human learning process, and imposes storage constraints and privacy concerns. Alternatively, continual learning has emerged as a crucial approach for developing unified and sustainable deep models to deal with new classes, tasks, and the drifting nature of data in non-stationary environments for various application areas. Continual learning techniques enable models to adapt and accumulate knowledge over time, which is essential for maintaining performance on evolving datasets and novel tasks. This systematic review paper provides a comprehensive overview of the state-of-the-art in continual learning techniques applied to medical imaging analysis. We present an extensive survey of existing research, covering topics including catastrophic forgetting, data drifts, stability, and plasticity requirements. Further, an in-depth discussion of key components of a continual learning framework such as continual learning scenarios, techniques, evaluation schemes, and metrics is provided. Continual learning techniques encompass various categories, including rehearsal, regularization, architectural, and hybrid strategies. We assess the popularity and applicability of continual learning categories in various medical sub-fields like radiology and histopathology...

Figures (8)

• Figure 1: A coarse level flowchart for designing a CL pipeline
• Figure 2: Ratio of CL-based research for downstream applications
• Figure 3: CL-based research contributions over the years. Percentages represent the number of CL papers in the medical domain each year, showing the increasing trend and growing importance of CL research.
• Figure 4: Ratio of CL-based works for different incremental scenarios
• Figure 5: Popularity of different CL strategies for medical image analysis