BiasPruner: Debiased Continual Learning for Medical Image Classification

TL;DR

BiasPruner addresses bias in medical image continual learning by intentionally forgetting spurious correlations to improve generalization and fairness. It introduces a bias score per network unit, $\mathcal{S}^t_{c,n}$, computed from easy (biased) and hard (unbiased) samples under $\mathcal{L}_{\mathrm{GCE}}$, and constructs per-task debiased subnetworks by pruning high-bias units with pruning ratio $\gamma$. Knowledge transfer is performed by pruning the full network to create a new task subnetwork that includes prior debiased subnetworks; previous subnetworks are frozen while new units learn. In experiments on three medical datasets (FITZ, HAM, NIH), BiasPruner consistently outperforms state-of-the-art continual learning methods in both accuracy and fairness without requiring dataset bias annotations, demonstrating practical impact for debiased continual learning in medicine.

Abstract

Continual Learning (CL) is crucial for enabling networks to dynamically adapt as they learn new tasks sequentially, accommodating new data and classes without catastrophic forgetting. Diverging from conventional perspectives on CL, our paper introduces a new perspective wherein forgetting could actually benefit the sequential learning paradigm. Specifically, we present BiasPruner, a CL framework that intentionally forgets spurious correlations in the training data that could lead to shortcut learning. Utilizing a new bias score that measures the contribution of each unit in the network to learning spurious features, BiasPruner prunes those units with the highest bias scores to form a debiased subnetwork preserved for a given task. As BiasPruner learns a new task, it constructs a new debiased subnetwork, potentially incorporating units from previous subnetworks, which improves adaptation and performance on the new task. During inference, BiasPruner employs a simple task-agnostic approach to select the best debiased subnetwork for predictions. We conduct experiments on three medical datasets for skin lesion classification and chest X-Ray classification and demonstrate that BiasPruner consistently outperforms SOTA CL methods in terms of classification performance and fairness. Our code is available here.

Figures (5)

• Figure 1: (Left) BiasPruner learns sequentially, allocating a subnetwork for each task. (Right) BiasPruner evaluates each network unit's contribution to learning spurious features from biased training data, assigning bias scores. High-score units are pruned, and the subnetwork is finetuned on both easy and hard samples.
• Figure 2: The overall (dashed) and DPR (dotted) performance of BiasPruner and other methods over all the seen tasks after each training step in the continual learning sequence, where $Ti$ refers to the $i$th task.
• Figure 3: Sensitive attribute detection from frozen models pre-trained to diagnose. BiasPruner low AUCs indicate that bias is not encoded in its resulting features.
• Figure 4: Visualization of easy (blue square) and hard (red square) samples across the different benchmarks. 1st row: Images from FITZ (Task 4), where each image is labeled by its Fitzpatrick skin tone, denoted as I, II, IV, V or VI. 2nd row: Images from HAM (Task 3), where each image is labeled by age (age $<$ or $\geq$ 60), denoted as $-$ or $+$, respectively. 3rd row: Images from NIH (Task 1), where each image is labeled as male or female, denoted as M or F, respectively. We notice that the hard samples represent the minority within their respective tasks.
• Figure 5: Bias distribution across the different tasks in FITZ, HAM and NIH.