Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Continual Deep Active Learning for Medical Imaging: Replay-Base Architecture for Context Adaptation

Rui Daniel, M. Rita Verdelho, Catarina Barata, Carlos Santiago

TL;DR

This paper proposes Replay-Based Architecture for Context Adaptation (RBACA), a continual active learning framework designed to enhance adaptation and generalization of deep medical imaging models across shifting contexts and limited annotations. RBACA combines memory-based rehearsal (continual learning) with an informativeness-driven active learning component to select diverse, informative samples within annotation budgets. It introduces the Incremental Learning Score (IL-Score) to jointly evaluate transfer learning, forgetting, and final performance, and demonstrates improvements in both cardiac segmentation (Domain-IL) and pathology classification (Class-IL) on the M&Ms cardiac imaging dataset. The work provides a flexible architecture with static/dynamic memory management, multiple pruning strategies, and class-incremental capabilities, contributing a practical approach to deploying robust medical imaging models in real-world, multi-center settings.

Abstract

Deep Learning for medical imaging faces challenges in adapting and generalizing to new contexts. Additionally, it often lacks sufficient labeled data for specific tasks requiring significant annotation effort. Continual Learning (CL) tackles adaptability and generalizability by enabling lifelong learning from a data stream while mitigating forgetting of previously learned knowledge. Active Learning (AL) reduces the number of required annotations for effective training. This work explores both approaches (CAL) to develop a novel framework for robust medical image analysis. Based on the automatic recognition of shifts in image characteristics, Replay-Base Architecture for Context Adaptation (RBACA) employs a CL rehearsal method to continually learn from diverse contexts, and an AL component to select the most informative instances for annotation. A novel approach to evaluate CAL methods is established using a defined metric denominated IL-Score, which allows for the simultaneous assessment of transfer learning, forgetting, and final model performance. We show that RBACA works in domain and class-incremental learning scenarios, by assessing its IL-Score on the segmentation and diagnosis of cardiac images. The results show that RBACA outperforms a baseline framework without CAL, and a state-of-the-art CAL method across various memory sizes and annotation budgets. Our code is available in https://github.com/RuiDaniel/RBACA .

Continual Deep Active Learning for Medical Imaging: Replay-Base Architecture for Context Adaptation

TL;DR

This paper proposes Replay-Based Architecture for Context Adaptation (RBACA), a continual active learning framework designed to enhance adaptation and generalization of deep medical imaging models across shifting contexts and limited annotations. RBACA combines memory-based rehearsal (continual learning) with an informativeness-driven active learning component to select diverse, informative samples within annotation budgets. It introduces the Incremental Learning Score (IL-Score) to jointly evaluate transfer learning, forgetting, and final performance, and demonstrates improvements in both cardiac segmentation (Domain-IL) and pathology classification (Class-IL) on the M&Ms cardiac imaging dataset. The work provides a flexible architecture with static/dynamic memory management, multiple pruning strategies, and class-incremental capabilities, contributing a practical approach to deploying robust medical imaging models in real-world, multi-center settings.

Abstract

Deep Learning for medical imaging faces challenges in adapting and generalizing to new contexts. Additionally, it often lacks sufficient labeled data for specific tasks requiring significant annotation effort. Continual Learning (CL) tackles adaptability and generalizability by enabling lifelong learning from a data stream while mitigating forgetting of previously learned knowledge. Active Learning (AL) reduces the number of required annotations for effective training. This work explores both approaches (CAL) to develop a novel framework for robust medical image analysis. Based on the automatic recognition of shifts in image characteristics, Replay-Base Architecture for Context Adaptation (RBACA) employs a CL rehearsal method to continually learn from diverse contexts, and an AL component to select the most informative instances for annotation. A novel approach to evaluate CAL methods is established using a defined metric denominated IL-Score, which allows for the simultaneous assessment of transfer learning, forgetting, and final model performance. We show that RBACA works in domain and class-incremental learning scenarios, by assessing its IL-Score on the segmentation and diagnosis of cardiac images. The results show that RBACA outperforms a baseline framework without CAL, and a state-of-the-art CAL method across various memory sizes and annotation budgets. Our code is available in https://github.com/RuiDaniel/RBACA .
Paper Structure (17 sections, 2 equations, 6 figures, 6 tables)

This paper contains 17 sections, 2 equations, 6 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Memory Management
  5. Memory Pruning
  6. Informativeness-Based Query Method
  7. Adaptation to Class-IL
  8. Experimental Setup
  9. M&Ms Dataset
  10. Tasks and Training Conditions
  11. Data Processing
  12. Evaluation
  13. Model Comparison
  14. Experimental Results and Discussion
  15. Cardiac Segmentation
  16. ...and 2 more sections

Figures (6)

  • Figure 1: The RBACA architecture. Processes a data stream $S$ and is composed of four key components: PC module, outlier memory $O$, training memory $M$, and task module. It supports various memory management (MM), pruning, and AL strategies.
  • Figure 2: RBACA, CASA and SeqFineTune segmentation IL-Score for multiple configurations. Points are labeled using the XYZ format, where X denotes either R or C, representing RBACA or CASA; Y indicates 1, 4, or 8, corresponding to $\beta$ values of 108, 430, or 860; and Z represents 1, 2, or 3, relating to $k$ values of 40, 97, or 400 in Dynamic MM, or to $K_M$ values of 200, 485, or 2000 in Static MM.
  • Figure 3: Image, target, and prediction agreement for a specific patient slice.
  • Figure 4: RBACA, CASA and SeqFineTune classification IL-Score for multiple configurations.
  • Figure 5: Pathology memory distribution in classification example.
  • ...and 1 more figures