Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Task-Agnostic Continual Learning for Chest Radiograph Classification

Muthu Subash Kavitha, Anas Zafar, Amgad Muneer, Jia Wu

TL;DR

The paper addresses the need for continual chest radiograph classifiers that can be updated with new datasets without retraining on old data or accessing historical images. It proposes CARL-XRay, a task-incremental framework that freezes a high-capacity backbone and adds lightweight task adapters and heads per dataset, plus a latent task selector with prototype memory and feature-level replay to stabilize routing under task-unknown inference. In experiments on MIMIC-CXR and CheXpert, CARL-XRay achieves strong diagnostic performance and reliable task-unknown routing (approximately 75% routing accuracy) while using orders of magnitude fewer trainable parameters than joint retraining. Ablation studies show that replay is essential for maintaining task identity, and the Continuum adapter offers the best balance of accuracy, routing reliability, and memory efficiency for continual deployment in clinical settings.

Abstract

Clinical deployment of chest radiograph classifiers requires models that can be updated as new datasets become available without retraining on previously ob- served data or degrading validated performance. We study, for the first time, a task-incremental continual learning setting for chest radiograph classification, in which heterogeneous chest X-ray datasets arrive sequentially and task identifiers are unavailable at inference. We propose a continual adapter-based routing learning strategy for Chest X-rays (CARL-XRay) that maintains a fixed high-capacity backbone and incrementally allocates lightweight task-specific adapters and classifier heads. A latent task selector operates on task-adapted features and leverages both current and historical context preserved through compact prototypes and feature-level experience replay. This design supports stable task identification and adaptation across sequential updates while avoiding raw-image storage. Experiments on large-scale public chest radiograph datasets demonstrate robust performance retention and reliable task-aware inference under continual dataset ingestion. CARL-XRay outperforms joint training under task-unknown deployment, achieving higher routing accuracy (75.0\% vs.\ 62.5\%), while maintaining competitive diagnostic performance with AUROC of 0.74 in the oracle setting with ground-truth task identity and 0.75 under task-unknown inference, using significantly fewer trainable parameters. Finally, the proposed framework provides a practical alternative to joint training and repeated full retraining in continual clinical deployment.

Task-Agnostic Continual Learning for Chest Radiograph Classification

TL;DR

The paper addresses the need for continual chest radiograph classifiers that can be updated with new datasets without retraining on old data or accessing historical images. It proposes CARL-XRay, a task-incremental framework that freezes a high-capacity backbone and adds lightweight task adapters and heads per dataset, plus a latent task selector with prototype memory and feature-level replay to stabilize routing under task-unknown inference. In experiments on MIMIC-CXR and CheXpert, CARL-XRay achieves strong diagnostic performance and reliable task-unknown routing (approximately 75% routing accuracy) while using orders of magnitude fewer trainable parameters than joint retraining. Ablation studies show that replay is essential for maintaining task identity, and the Continuum adapter offers the best balance of accuracy, routing reliability, and memory efficiency for continual deployment in clinical settings.

Abstract

Clinical deployment of chest radiograph classifiers requires models that can be updated as new datasets become available without retraining on previously ob- served data or degrading validated performance. We study, for the first time, a task-incremental continual learning setting for chest radiograph classification, in which heterogeneous chest X-ray datasets arrive sequentially and task identifiers are unavailable at inference. We propose a continual adapter-based routing learning strategy for Chest X-rays (CARL-XRay) that maintains a fixed high-capacity backbone and incrementally allocates lightweight task-specific adapters and classifier heads. A latent task selector operates on task-adapted features and leverages both current and historical context preserved through compact prototypes and feature-level experience replay. This design supports stable task identification and adaptation across sequential updates while avoiding raw-image storage. Experiments on large-scale public chest radiograph datasets demonstrate robust performance retention and reliable task-aware inference under continual dataset ingestion. CARL-XRay outperforms joint training under task-unknown deployment, achieving higher routing accuracy (75.0\% vs.\ 62.5\%), while maintaining competitive diagnostic performance with AUROC of 0.74 in the oracle setting with ground-truth task identity and 0.75 under task-unknown inference, using significantly fewer trainable parameters. Finally, the proposed framework provides a practical alternative to joint training and repeated full retraining in continual clinical deployment.
Paper Structure (30 sections, 10 equations, 3 figures, 5 tables, 1 algorithm)

This paper contains 30 sections, 10 equations, 3 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Continual Learning and Parameter Isolation
  4. Task Awareness and Modular Continual Updates
  5. Methodology
  6. Problem Setting
  7. Model Architecture
  8. Training Objective and Label Handling
  9. Latent Task Selector with Prototype Memory
  10. Task unknown Inference and Routing
  11. Experimental Settings and Results
  12. Evaluating Diagnostic Performance
  13. Evaluation Metrics.
  14. Task Definition.
  15. Sequential Continual Learning Performance.
  16. ...and 15 more sections

Figures (3)

  • Figure 1: Proposed continual learning framework.
  • Figure 2: Diagnostic performance of CARL-XRay. (a) Forgetting on Task 1 is 0.012, indicating strong retention under sequential updates. (b) Under task unknown inference, sequential training achieves 75% routing accuracy by maintaining task specific structure through isolated adapters and feature level replay. In contrast, joint training attains 62.5% routing accuracy, suggesting weaker task separability when both datasets are optimized simultaneously.
  • Figure 3: Task-unknown inference analysis. (a) Comparison of oracle (task-known) and routed (task-unknown) diagnostic performance on MIMIC and CheXpert. (b) Confusion matrix showing task routing accuracy between MIMIC and CheXpert under task-unknown inference.