Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Uncertainty propagation through trained multi-layer perceptrons: Exact analytical results

Andrew Thompson, Miles McCrory

TL;DR

This paper derives exact analytical expressions for propagating uncertainty through a trained single-hidden-layer MLP with ReLU activations when the input is multivariate Gaussian. By obtaining closed-form moments of the rectified Gaussian hidden activations—specifically $\mathbb{E}[X_i]$, $\mathbb{E}[X_i^2]$, and $\mathbb{E}[X_i X_j]$—the authors compute the output moments $E[Y]$ and $Var[Y]$ exactly as $E[Y] = \beta^T \gamma + d$ and $Var[Y] = \beta^T \Gamma \beta$. These results rely on standard univariate and bivariate Gaussian integrals and avoid infinite series, in contrast to prior work. The analytical expressions are validated against Monte Carlo simulations on an EIS-based state-of-health (SOH) prediction problem, demonstrating convergence consistent with $1/\sqrt{n}$. The work offers a transparent, accurate, and reproducible method for uncertainty quantification in regression models and discusses extensions to other activations and deeper networks as future directions.

Abstract

We give analytical results for propagation of uncertainty through trained multi-layer perceptrons (MLPs) with a single hidden layer and ReLU activation functions. More precisely, we give expressions for the mean and variance of the output when the input is multivariate Gaussian. In contrast to previous results, we obtain exact expressions without resort to a series expansion.

Uncertainty propagation through trained multi-layer perceptrons: Exact analytical results

TL;DR

This paper derives exact analytical expressions for propagating uncertainty through a trained single-hidden-layer MLP with ReLU activations when the input is multivariate Gaussian. By obtaining closed-form moments of the rectified Gaussian hidden activations—specifically , , and —the authors compute the output moments and exactly as and . These results rely on standard univariate and bivariate Gaussian integrals and avoid infinite series, in contrast to prior work. The analytical expressions are validated against Monte Carlo simulations on an EIS-based state-of-health (SOH) prediction problem, demonstrating convergence consistent with . The work offers a transparent, accurate, and reproducible method for uncertainty quantification in regression models and discusses extensions to other activations and deeper networks as future directions.

Abstract

We give analytical results for propagation of uncertainty through trained multi-layer perceptrons (MLPs) with a single hidden layer and ReLU activation functions. More precisely, we give expressions for the mean and variance of the output when the input is multivariate Gaussian. In contrast to previous results, we obtain exact expressions without resort to a series expansion.
Paper Structure (11 sections, 7 theorems, 36 equations, 2 figures, 1 table)

This paper contains 11 sections, 7 theorems, 36 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Paper structure
  3. Related work
  4. Neural network architecture
  5. Analytical uncertainty propagation results
  6. Validation of the analytical expressions
  7. Conclusions
  8. Proof of Theorem \ref{['thrm:xixj']}
  9. Extension of analytical results to corner cases
  10. The case of $\sigma_i=0$ or $\sigma_j=0$
  11. The case of $|\rho_{ij}|=1$

Key Result

Theorem 1

Let $X$ be defined as above and suppose $\sigma_i>0$. Then, for all $i=1,\ldots,p$, and

Figures (2)

  • Figure 1: Architecture of an MLP with a single hidden layer and ReLU activation function.
  • Figure 2: Log-log plots of RMSE against number of Monte Carlo samples.

Theorems & Definitions (8)

  • Theorem 1: frey1999variational
  • Theorem 2
  • Lemma 1
  • proof
  • Lemma 2: mood1950introduction
  • Lemma 3
  • Lemma 4
  • Theorem 3