Continual Learning in Medical Imaging: A Survey and Practical Analysis

TL;DR

This survey comprehensively review the recent literature on continual learning in the medical domain, highlight recent trends, and point out several practical issues and develops a taxonomy for the reviewed studies.

Abstract

Deep Learning has shown great success in reshaping medical imaging, yet it faces numerous challenges hindering widespread application. Issues like catastrophic forgetting and distribution shifts in the continuously evolving data stream increase the gap between research and applications. Continual Learning offers promise in addressing these hurdles by enabling the sequential acquisition of new knowledge without forgetting previous learnings in neural networks. In this survey, we comprehensively review the recent literature on continual learning in the medical domain, highlight recent trends, and point out the practical issues. Specifically, we survey the continual learning studies on classification, segmentation, detection, and other tasks in the medical domain. Furthermore, we develop a taxonomy for the reviewed studies, identify the challenges, and provide insights to overcome them. We also critically discuss the current state of continual learning in medical imaging, including identifying open problems and outlining promising future directions. We hope this survey will provide researchers with a useful overview of the developments in the field and will further increase interest in the community. To keep up with the fast-paced advancements in this field, we plan to routinely update the repository with the latest relevant papers at https://github.com/BioMedIA-MBZUAI/awesome-cl-in-medical .

Figures (9)

• Figure 1: The different changes through time that can occur in a healthcare scenario. The changes are depicted horizontally, with the time flow depicted vertically. The aim in CL is for the AI model depicted in the center of the figure to adapt to these data changes through time. Updates can include (i) a new center, (ii) a new modality, and/or (iii) a new task
• Figure 2: Trend of studies in Continual Learning for Medical Imaging between 2018 and 2023 with the current and projected studies from 2024.
• Figure 3: The Three Types of Continual Learning. While Class-Incremental Learning and Task-Incremental Learning share training methodologies, Task-Incremental Learning requires task ID at inference. Meanwhile, Domain-Incremental Learning maintains consistent labels across tasks with varying data distributions.
• Figure 4: Taxonomy of Continual Learning Approaches.
• Figure 5: Regularization Based Approach: The model is updated by constraining the parameter changes. Assuming $\theta_{new}$ and $\theta_{old}$ are the parameters of the updated and previous models, respectively. The objective is to find an optimal point between the parameter space of these models.