Batch Selection for Multi-Label Classification Guided by Uncertainty and Dynamic Label Correlations

TL;DR

This work tackles batch construction in multi-label learning by introducing an uncertainty-based batch selection that jointly leverages per-label uncertainty and dynamic label correlations. It defines per-label uncertainty as a blend of current prediction confidence and recent prediction fluctuations, and constructs a mutual-information-based label-correlation matrix to weight sample uncertainties, yielding an uncertainty-driven sampling distribution with decaying selection pressure to promote diversity. Empirical results across multiple datasets and three deep multi-label models show consistent improvements in Macro-AUC, along with faster convergence and favorable ranking and Hamming losses, supported by statistical tests. The approach is robust across domains and model types, with extensions suggested toward temperature-scaled entropy and Bayesian uncertainty methods for further enhancement.

Abstract

The accuracy of deep neural networks is significantly influenced by the effectiveness of mini-batch construction during training. In single-label scenarios, such as binary and multi-class classification tasks, it has been demonstrated that batch selection algorithms preferring samples with higher uncertainty achieve better performance than difficulty-based methods. Although there are two batch selection methods tailored for multi-label data, none of them leverage important uncertainty information. Adapting the concept of uncertainty to multi-label data is not a trivial task, since there are two issues that should be tackled. First, traditional variance or entropy-based uncertainty measures ignore fluctuations of predictions within sliding windows and the importance of the current model state. Second, existing multi-label methods do not explicitly exploit the label correlations, particularly the uncertainty-based label correlations that evolve during the training process. In this paper, we propose an uncertainty-based multi-label batch selection algorithm. It assesses uncertainty for each label by considering differences between successive predictions and the confidence of current outputs, and further leverages dynamic uncertainty-based label correlations to emphasize instances whose uncertainty is synergistically expressed across multiple labels. Empirical studies demonstrate the effectiveness of our method in improving the performance and accelerating the convergence of various multi-label deep learning models.

Figures (11)

• Figure 1: Sample 1 containing "Messi," "World Cup," and "Adidas Golden Ball,"
• Figure 2: The different uncertainty measurement methods across three cases for historical predictions of the label.
• Figure 3: If we directly add up the uncertainty of each label, the total uncertainty of the first sample is equal to that of the second sample. However, after introducing the mutual information matrix $\mathbf{C}$, the recalculated uncertainty matrix shows that under the influence of the first mutual information matrix, the total uncertainty of the first sample (3.89) is greater than the uncertainty of the second sample (3.10), whereas under the influence of the second mutual information matrix, the total uncertainty of the first sample (3.00) is less than the uncertainty of the second sample (3.89).
• Figure 4: The convergence curves of five batch selection methods using CLIF.
• Figure 5: The Macro-AUC on validation set of five batch selection methods using CLIF.