Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cubature-based uncertainty estimation for nonlinear regression models

Martin Bubel, Jochen Schmid, Maximilian Carmesin, Volodymyr Kozachynskyi, Erik Esche, Michael Bortz

TL;DR

This work uses cubature formulas based on sparse grids to calculate the variance of the regression results of the NRTL model, calibrated to observations from different experimental designs, and presents exact benchmark results.

Abstract

Calibrating model parameters to measured data by minimizing loss functions is an important step in obtaining realistic predictions from model-based approaches, e.g., for process optimization. This is applicable to both knowledge-driven and data-driven model setups. Due to measurement errors, the calibrated model parameters also carry uncertainty. In this contribution, we use cubature formulas based on sparse grids to calculate the variance of the regression results. The number of cubature points is close to the theoretical minimum required for a given level of exactness. We present exact benchmark results, which we also compare to other cubatures. This scheme is then applied to estimate the prediction uncertainty of the NRTL model, calibrated to observations from different experimental designs.

Cubature-based uncertainty estimation for nonlinear regression models

TL;DR

This work uses cubature formulas based on sparse grids to calculate the variance of the regression results of the NRTL model, calibrated to observations from different experimental designs, and presents exact benchmark results.

Abstract

Calibrating model parameters to measured data by minimizing loss functions is an important step in obtaining realistic predictions from model-based approaches, e.g., for process optimization. This is applicable to both knowledge-driven and data-driven model setups. Due to measurement errors, the calibrated model parameters also carry uncertainty. In this contribution, we use cubature formulas based on sparse grids to calculate the variance of the regression results. The number of cubature points is close to the theoretical minimum required for a given level of exactness. We present exact benchmark results, which we also compare to other cubatures. This scheme is then applied to estimate the prediction uncertainty of the NRTL model, calibrated to observations from different experimental designs.
Paper Structure (19 sections, 6 theorems, 115 equations, 21 figures, 1 table)

This paper contains 19 sections, 6 theorems, 115 equations, 21 figures, 1 table.

Table of Contents

  1. Introduction
  2. Approximating the prediction uncertainty
  3. Approximation using Monte Carlo sampling
  4. Approximation using linearization
  5. Approximations using cubature formulas
  6. Cubature formula based on sigma points
  7. Cubature formula of McNamee and Stenger
  8. Cubature formula of Lu and Darmofal
  9. Validation and benchmarking
  10. Validation on a generic quadratic model
  11. Validation on more general models and designs
  12. Quadratic model in one input dimension
  13. Quadratic model in two input dimensions
  14. Exponential growth model
  15. NRTL model
  16. ...and 4 more sections

Key Result

Lemma 1

Suppose $f$ is the generic quadratic model defined by eq:multivariatetoymodel and $\tilde{x} = (x_1,\dots,x_n) \in \mathcal{X}$ is an experimental design satisfying eq:multivariatetoydesigncondition1 and eq:multivariatetoydesigncondition2. Then for every $\tilde{y} = (y_1,\dots,y_n) \in \mathbb{R}^n for all $k \in \{1,\dots,d_{x}\}$. Additionally, the estimated model based on $(\tilde{x},\tilde{y}

Figures (21)

  • Figure 1: Approximation error \ref{['eq:validationloss']} for the quadratic model \ref{['eq:multivariatetoymodel']} in $d_{x} = 2$ input dimensions with coefficients \ref{['eq:model-coefficients-validation']} and with the factorial design \ref{['eq:factorial-design-for-validation']}. The approximation error is plotted as a function $x_1 \mapsto \Delta_{\tilde{x}, \tilde{y}}^{\mathrm{method}}\left(x_1, x_2\right)$ of $x_1$ for several fixed values of $x_2$ and for $\mathrm{method} = \text{LIN}$ (blue circles) and $\mathrm{method} = \text{LD}$ (lime squares).
  • Figure 2: Model and observations of the univariate quadratic model for the factorial (A) and equidistant (B) design. The design points are marked by the red circles and the true model is shown as the blue line. The vertical axes of both subplots (A) and (B) are aligned and have the same scale.
  • Figure 3: Convergence of the MC approximator $V_{\tilde{x}}^{\text{MC}}(x)$ to the prediction uncertainty $V_{\tilde{x}}(x)$ of the univariate quadratic model at $x := 0$ over the number of samples for the factorial (A) and equidistant (B) design.
  • Figure 4: Least-squares estimates for $10^3$ from the $N_{\text{MC}} := 10^6$ Monte-Carlo samples of the univariate quadratic model for the factorial (A) and equidistant (B) design. The MC samples are marked by the blue circles, the true parameter value $\theta^*$ is marked by the lime pentagon, and the MC mean estimator $\bar{\theta}$ is marked by the yellow square. The vertical axes of both subplots (A) and (B) are aligned and have the same scale.
  • Figure 5: Histrogram of parameter estimation errors \ref{['eq:parameterestimatorerror']} of the univariate quadratic model for the factorial (A) and equidistant (B) design. The vertical axes of both subplots (A) and (B) are aligned and have the same scale.
  • ...and 16 more figures

Theorems & Definitions (12)

  • Lemma 1
  • proof
  • Corollary 2
  • proof
  • Lemma 3
  • proof
  • Proposition 4
  • proof
  • Proposition 5
  • proof
  • ...and 2 more