Trust the Unreliability: Inward Backward Dynamic Unreliability Driven Coreset Selection for Medical Image Classification

Abstract

Efficiently managing and utilizing large-scale medical imaging datasets with limited resources presents significant challenges. While coreset selection helps reduce computational costs, its effectiveness in medical data remains limited due to inherent complexity, such as large intra-class variation and high inter-class similarity. To address this, we revisit the training process and observe that neural networks consistently produce stable confidence predictions and better remember samples near class centers in training. However, concentrating on these samples may complicate the modeling of decision boundaries. Hence, we argue that the more unreliable samples are, in fact, the more informative in helping build the decision boundary. Based on this, we propose the Dynamic Unreliability-Driven Coreset Selection(DUCS) strategy. Specifically, we introduce an inward-backward unreliability assessment perspective: 1) Inward Self-Awareness: The model introspects its behavior by analyzing the evolution of confidence during training, thereby quantifying uncertainty of each sample. 2) Backward Memory Tracking: The model reflects on its training tracking by tracking the frequency of forgetting samples, thus evaluating its retention ability for each sample. Next, we select unreliable samples that exhibit substantial confidence fluctuations and are repeatedly forgotten during training. This selection process ensures that the chosen samples are near the decision boundary, thereby aiding the model in refining the boundary. Extensive experiments on public medical datasets demonstrate our superior performance compared to state-of-the-art(SOTA) methods, particularly at high compression rates.

Figures (7)

• Figure 1: Examples of intra-class variation and inter-class similarity in medical image classification.
• Figure 1: The main steps of the proposed DUCS.
• Figure 2: The framework of our proposed DUCS. DUCS is built upon a novel inward-backward unreliability assessment perspective. (1) To look inward at the model’s self-awareness, we parameterize the final layer outputs as concentration parameters of a Dirichlet distribution to quantify the epistemic uncertainty. This allows us to compute an epistemic uncertainty score $S_{t}^{i}$ for each sample during training and subsequently calculate its variance $V_t^{i}$ over a sliding window of epochs. (2) To look backward at the model's memory tracking, we calculate the forgetting frequency $F_t^{i}$ throughout the training history. Ultimately, the coreset is constructed by selecting samples that exhibit both high variance in epistemic uncertainty and high forgetting frequency.
• Figure 3: t-SNE visualizations. "$\CIRCLE$" and "$\blacktriangle$" denote top 20% unreliable and reliable samples.
• Figure 4: The score variations of two randomly selected samples from OrganSMNIST and OrganAMNIST during the training process. Different colors represent different samples.