Confidence-Based Task Prediction in Continual Disease Classification Using Probability Distribution

TL;DR

This work tackles the challenge of unknown task identities in continual disease classification while respecting privacy constraints. It introduces Confidence-Based Task Prediction (CTP), an exemplar-free approach that uses per-task experts and logits-based comparisons to infer the correct task at inference, aided by a DisMax loss that promotes high-entropy distributions for data from other tasks. A noise-region defined in the logits distribution enables robust confidence scoring, enabling accurate task-id prediction and subsequent classification. Experimental results on PathMNIST and OCT show CTP outperforms several continual learning baselines, with further gains when a data continuum is available, achieving near joint-training performance in such scenarios and demonstrating practical potential for clinical deployment.

Abstract

Deep learning models are widely recognized for their effectiveness in identifying medical image findings in disease classification. However, their limitations become apparent in the dynamic and ever-changing clinical environment, characterized by the continuous influx of newly annotated medical data from diverse sources. In this context, the need for continual learning becomes particularly paramount, not only to adapt to evolving medical scenarios but also to ensure the privacy of healthcare data. In our research, we emphasize the utilization of a network comprising expert classifiers, where a new expert classifier is added each time a new task is introduced. We present CTP, a task-id predictor that utilizes confidence scores, leveraging the probability distribution (logits) of the classifier to accurately determine the task-id at inference time. Logits are adjusted to ensure that classifiers yield a high-entropy distribution for data associated with tasks other than their own. By defining a noise region in the distribution and computing confidence scores, CTP achieves superior performance when compared to other relevant continual learning methods. Additionally, the performance of CTP can be further improved by providing it with a continuum of data at the time of inference.

Figures (4)

• Figure 1: \ref{['fig:dist']} The correct classifier ($f_2$) for the PathMNIST image produces high-value logits for the correct class and low-value logits for the incorrect classes. The logits from all the classifier outputs are combined, and a noise region (in red) is established. The confidence score for each classifier is computed based on the number of logits from that classifier falling within the defined noise region. \ref{['fig:noise']} Noise region is determined based on histogram of the combined logits. $\beta$ percent of logits are chosen from both high-value and low-value logits to define the noise region.
• Figure 2: Average classification accuracy as training progresses.
• Figure 3: Final average task prediction accuracy and classification accuracy for different sizes of data continuum.
• Figure 4: Classification accuracies across various combinations of $\alpha$ and $\beta$ values.