Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Analytical results for uncertainty propagation through trained machine learning regression models

Andrew Thompson

TL;DR

This work develops analytical expressions for propagating input uncertainty through fixed regression ML models, covering linear, ridge, SVM and four kernel-based methods (kernel ridge, Gaussian Processes, kernel SVM, and Relevance Vector Machines). By deriving mean and variance formulas for predictions under various input distributions (Gaussian, uniform, symmetric triangular) and kernel choices (notably the RBF kernel), the paper demonstrates when analytical propagation outperforms Monte Carlo sampling in both accuracy and efficiency. The authors validate their framework with a lithium-ion battery state-of-health case study using EIS data, showing close agreement with Monte Carlo results and highlighting computational advantages in metrology contexts. Limitations include the focus on fixed models and RBF kernels, with future work aimed at numerical stability, extending to additional model families, and incorporating model uncertainty via variance decomposition.

Abstract

Machine learning (ML) models are increasingly being used in metrology applications. However, for ML models to be credible in a metrology context they should be accompanied by principled uncertainty quantification. This paper addresses the challenge of uncertainty propagation through trained/fixed machine learning (ML) regression models. Analytical expressions for the mean and variance of the model output are obtained/presented for certain input data distributions and for a variety of ML models. Our results cover several popular ML models including linear regression, penalised linear regression, kernel ridge regression, Gaussian Processes (GPs), support vector machines (SVMs) and relevance vector machines (RVMs). We present numerical experiments in which we validate our methods and compare them with a Monte Carlo approach from a computational efficiency point of view. We also illustrate our methods in the context of a metrology application, namely modelling the state-of-health of lithium-ion cells based upon Electrical Impedance Spectroscopy (EIS) data

Analytical results for uncertainty propagation through trained machine learning regression models

TL;DR

This work develops analytical expressions for propagating input uncertainty through fixed regression ML models, covering linear, ridge, SVM and four kernel-based methods (kernel ridge, Gaussian Processes, kernel SVM, and Relevance Vector Machines). By deriving mean and variance formulas for predictions under various input distributions (Gaussian, uniform, symmetric triangular) and kernel choices (notably the RBF kernel), the paper demonstrates when analytical propagation outperforms Monte Carlo sampling in both accuracy and efficiency. The authors validate their framework with a lithium-ion battery state-of-health case study using EIS data, showing close agreement with Monte Carlo results and highlighting computational advantages in metrology contexts. Limitations include the focus on fixed models and RBF kernels, with future work aimed at numerical stability, extending to additional model families, and incorporating model uncertainty via variance decomposition.

Abstract

Machine learning (ML) models are increasingly being used in metrology applications. However, for ML models to be credible in a metrology context they should be accompanied by principled uncertainty quantification. This paper addresses the challenge of uncertainty propagation through trained/fixed machine learning (ML) regression models. Analytical expressions for the mean and variance of the model output are obtained/presented for certain input data distributions and for a variety of ML models. Our results cover several popular ML models including linear regression, penalised linear regression, kernel ridge regression, Gaussian Processes (GPs), support vector machines (SVMs) and relevance vector machines (RVMs). We present numerical experiments in which we validate our methods and compare them with a Monte Carlo approach from a computational efficiency point of view. We also illustrate our methods in the context of a metrology application, namely modelling the state-of-health of lithium-ion cells based upon Electrical Impedance Spectroscopy (EIS) data
Paper Structure (32 sections, 7 theorems, 64 equations, 4 figures, 2 tables)

This paper contains 32 sections, 7 theorems, 64 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Uncertainty quantification for ML models
  3. Analytical approaches for uncertainty quantification
  4. Our contribution
  5. Paper structure
  6. Notation
  7. Related work
  8. Linear models
  9. Background on linear models
  10. Uncertainty propagation results
  11. Kernel-based models
  12. The general framework
  13. Background on four kernel-based methods
  14. Kernel ridge regression
  15. Gaussian Processes
  16. ...and 17 more sections

Key Result

Theorem 1

Let ${ \mathsfbi{X} }$ be such that $\mathbb{E}\, { \mathsfbi{X} } = { \mathsfbi{\mu} }$ and $\mathrm{Cov}\, { \mathsfbi{X} } = { \mathsfbi{\Gamma} }$ and suppose a prediction $Y^{\ast}$ is obtained from the linear model (linear_model). Then

Figures (4)

  • Figure 1: Plots of predicted SOH against recorded SOH on one instantiation of the test set.
  • Figure 2: Visualisation of linear and GP models for SOH.
  • Figure 3: Dependence of running time of the analytical and Monte Carlo sampling approaches upon number of input variables and number of Monte Carlo samples for a linear model.
  • Figure 4: (a) GP models for $10^2$ and $10^4$ training samples; (b) Dependence of running time of the analytical and Monte Carlo sampling approaches upon number of training samples and number of Monte Carlo samples for a GP model.

Theorems & Definitions (7)

  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Theorem 4: quinonero2003prediction
  • Theorem 5: wan2014analytical
  • Theorem 6: wan2014analytical
  • Theorem 7