Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Uncertainty quantification in neural network classifiers -- a local linear approach

Magnus Malmström, Isaac Skog, Daniel Axehill, Fredrik Gustafsson

TL;DR

A method to estimate the probability mass function (PMF) of the different classes, as well as the covariance of the estimated PMF, and proper risk assessment and fusion of multiple classifiers is proposed.

Abstract

Classifiers based on neural networks (NN) often lack a measure of uncertainty in the predicted class. We propose a method to estimate the probability mass function (PMF) of the different classes, as well as the covariance of the estimated PMF. First, a local linear approach is used during the training phase to recursively compute the covariance of the parameters in the NN. Secondly, in the classification phase another local linear approach is used to propagate the covariance of the learned NN parameters to the uncertainty in the output of the last layer of the NN. This allows for an efficient Monte Carlo (MC) approach for: (i) estimating the PMF; (ii) calculating the covariance of the estimated PMF; and (iii) proper risk assessment and fusion of multiple classifiers. Two classical image classification tasks, i.e., MNIST, and CFAR10, are used to demonstrate the efficiency the proposed method.

Uncertainty quantification in neural network classifiers -- a local linear approach

TL;DR

A method to estimate the probability mass function (PMF) of the different classes, as well as the covariance of the estimated PMF, and proper risk assessment and fusion of multiple classifiers is proposed.

Abstract

Classifiers based on neural networks (NN) often lack a measure of uncertainty in the predicted class. We propose a method to estimate the probability mass function (PMF) of the different classes, as well as the covariance of the estimated PMF. First, a local linear approach is used during the training phase to recursively compute the covariance of the parameters in the NN. Secondly, in the classification phase another local linear approach is used to propagate the covariance of the learned NN parameters to the uncertainty in the output of the last layer of the NN. This allows for an efficient Monte Carlo (MC) approach for: (i) estimating the PMF; (ii) calculating the covariance of the estimated PMF; and (iii) proper risk assessment and fusion of multiple classifiers. Two classical image classification tasks, i.e., MNIST, and CFAR10, are used to demonstrate the efficiency the proposed method.
Paper Structure (26 sections, 46 equations, 2 figures, 1 table)

This paper contains 26 sections, 46 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Problem formulation and contributions
  3. Parameter estimation
  4. Prediction and classification
  5. Temperature scaling
  6. Marginalization of parameter uncertainties
  7. Challenges and contributions
  8. Posterior parameter distribution
  9. Laplace approximation
  10. Asymptotic distribution
  11. Recursive computation of covariance
  12. Approximating the covariance
  13. Efficient MC sampling
  14. Marginalization using the delta method
  15. Fusion
  16. ...and 11 more sections

Figures (2)

  • Figure 1: Example of classification using \ref{['eq:gmarg']}. Left: inputs $x^\circ_n$. Middle: Ellipse representation of $P^{g}_N$, samples $g^{(k)}(x^\circ_n)$ and decision line between the classes representing 7 and 9. Right: Estimated $\hat{f}(x^\circ|\mathcal{T})$.
  • Figure 2: Reliability diagrams for prediction on the dataset. The diagrams illustrate the six different methods to measure the confidence in the prediction described in \ref{['sec:experiment_result']}. A calibration line is also shown in black.