Awareness of uncertainty in classification using a multivariate model and multi-views

TL;DR

This work introduces an uncertainty-aware negative log-likelihood loss (UANLL) for multiclass classification with spherical covariance, enabling simultaneous prediction of class scores and predictive uncertainty. It couples this loss with label smoothing and an at-test data-augmentation strategy that yields multiple predictions per sample, which are combined via several multi-view weighting schemes and optionally tuned with particle swarm optimization. Experiments on CIFAR-10 with asymmetric label noise show that UANLL-based models, especially when using multi-view predictions (MVW variants), achieve competitive accuracy and calibration compared to several uncertainty-estimation baselines. The approach highlights practical benefits of modeling data uncertainty in classification and lays groundwork for extending to out-of-distribution robustness and broader tasks.

Abstract

One of the ways to make artificial intelligence more natural is to give it some room for doubt. Two main questions should be resolved in that way. First, how to train a model to estimate uncertainties of its own predictions? And then, what to do with the uncertain predictions if they appear? First, we proposed an uncertainty-aware negative log-likelihood loss for the case of N-dimensional multivariate normal distribution with spherical variance matrix to the solution of N-classes classification tasks. The loss is similar to the heteroscedastic regression loss. The proposed model regularizes uncertain predictions, and trains to calculate both the predictions and their uncertainty estimations. The model fits well with the label smoothing technique. Second, we expanded the limits of data augmentation at the training and test stages, and made the trained model to give multiple predictions for a given number of augmented versions of each test sample. Given the multi-view predictions together with their uncertainties and confidences, we proposed several methods to calculate final predictions, including mode values and bin counts with soft and hard weights. For the latter method, we formalized the model tuning task in the form of multimodal optimization with non-differentiable criteria of maximum accuracy, and applied particle swarm optimization to solve the tuning task. The proposed methodology was tested using CIFAR-10 dataset with clean and noisy labels and demonstrated good results in comparison with other uncertainty estimation methods related to sample selection, co-teaching, and label smoothing.

Figures (3)

• Figure 1: The proposed UANLL loss intuition: the model inputs an image sample (or its fragment) and outputs two values: a vector of predictions $\mathbf{h}$ with the components $h_k \in (0, 1)$, $\sum{h_k} = 1$, which should be close to the one-hot encoded label vector $\mathbf{y}$, and a scalar value of uncertainty estimation $s \in (0, \inf)$, and then the loss term is calculated in correspondence with \ref{['eq:UANLL_2']}.
• Figure 2: Preliminary experiments demonstrate dependence of the models accuracy on the number of multi-view predictions (a), on the random crop scale $sc$ (b), and on the threshold value for the MVWCo-H and MVWCe-H methods (c).
• Figure 3: Validation loss and accuracy of the models based on the proposed loss \ref{['eq:UANLL_2']} trained with clean labels (a), and labels with asymmetric noise of 20% (b), and 40% (see \ref{['tab:RnResults']}).