Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hyper-differential sensitivity analysis with respect to model discrepancy: Prior distributions

Joseph Hart, Bart van Bloemen Waanders, Jixian Li, Timbwaoga A. J. Ouermi, Chris R. Johnson

TL;DR

This work targets uncertainty quantification in PDE-constrained optimization when high-fidelity evaluations are limited. It introduces a two-part framework for Hyper-differential sensitivity analysis with respect to model discrepancy (HDSA-MD): algorithmic initialization of prior hyper-parameters and a visualization framework to analyze prior samples and guide tuning, ensuring the discrepancy prior captures realistic uncertainty. By formulating the discrepancy as an affine operator ${\updelta}(\boldsymbol{z},\boldsymbol{\uptheta})$ and adopting a Gaussian prior on the parameter ${\boldsymbol{\uptheta}}$, the authors derive a well-posed, tractable posterior and demonstrate how to compute prior samples efficiently via Kronecker-structured covariances and trace-class operators. Numerical experiments across stationary and transient problems illustrate robustness of initialization, the role of temporal weighting, and the necessity of visual verification to avoid overconfident or overly diffuse posteriors. The proposed methodology offers scalable, physics-informed prior tuning and practical uncertainty quantification for multifidelity optimization with limited high-fidelity runs.

Abstract

Hyper-differential sensitivity analysis with respect to model discrepancy was recently developed to enable uncertainty quantification for optimization problems. The approach consists of two primary steps: (i) Bayesian calibration of the discrepancy between high- and low-fidelity models, and (ii) propagating the model discrepancy uncertainty through the optimization problem. When high-fidelity model evaluations are limited, as is common in practice, the prior discrepancy distribution plays a crucial role in the uncertainty analysis. However, specification of this prior is challenging due to its mathematical complexity and many hyper-parameters. This article presents a novel approach to specify the prior distribution. Our approach consists of two parts: (1) an algorithmic initialization of the prior hyper-parameters that uses existing data to initialize a hyper-parameter estimate, and (2) a visualization framework to systematically explore properties of the prior and guide tuning of the hyper-parameters to ensure that the prior captures the appropriate range of uncertainty. We provide detailed mathematical analysis and a collection of numerical examples that elucidate properties of the prior that are crucial to ensure uncertainty quantification.

Hyper-differential sensitivity analysis with respect to model discrepancy: Prior distributions

TL;DR

This work targets uncertainty quantification in PDE-constrained optimization when high-fidelity evaluations are limited. It introduces a two-part framework for Hyper-differential sensitivity analysis with respect to model discrepancy (HDSA-MD): algorithmic initialization of prior hyper-parameters and a visualization framework to analyze prior samples and guide tuning, ensuring the discrepancy prior captures realistic uncertainty. By formulating the discrepancy as an affine operator and adopting a Gaussian prior on the parameter , the authors derive a well-posed, tractable posterior and demonstrate how to compute prior samples efficiently via Kronecker-structured covariances and trace-class operators. Numerical experiments across stationary and transient problems illustrate robustness of initialization, the role of temporal weighting, and the necessity of visual verification to avoid overconfident or overly diffuse posteriors. The proposed methodology offers scalable, physics-informed prior tuning and practical uncertainty quantification for multifidelity optimization with limited high-fidelity runs.

Abstract

Hyper-differential sensitivity analysis with respect to model discrepancy was recently developed to enable uncertainty quantification for optimization problems. The approach consists of two primary steps: (i) Bayesian calibration of the discrepancy between high- and low-fidelity models, and (ii) propagating the model discrepancy uncertainty through the optimization problem. When high-fidelity model evaluations are limited, as is common in practice, the prior discrepancy distribution plays a crucial role in the uncertainty analysis. However, specification of this prior is challenging due to its mathematical complexity and many hyper-parameters. This article presents a novel approach to specify the prior distribution. Our approach consists of two parts: (1) an algorithmic initialization of the prior hyper-parameters that uses existing data to initialize a hyper-parameter estimate, and (2) a visualization framework to systematically explore properties of the prior and guide tuning of the hyper-parameters to ensure that the prior captures the appropriate range of uncertainty. We provide detailed mathematical analysis and a collection of numerical examples that elucidate properties of the prior that are crucial to ensure uncertainty quantification.
Paper Structure (26 sections, 5 theorems, 66 equations, 8 figures, 1 algorithm)

This paper contains 26 sections, 5 theorems, 66 equations, 8 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Formulation and Background
  3. Post-optimality sensitivity with respect to model discrepancy
  4. Model discrepancy parameterization
  5. Bayesian discrepancy calibration
  6. Noise model
  7. Prior distribution
  8. Trace class covariance operators
  9. Computing prior discrepancy samples
  10. hyper-parameter specification
  11. Smoothness hyper-parameters
  12. Temporal variance weighting hyper-parameter
  13. State variance hyper-parameter
  14. Optimization variable variance hyper-parameter
  15. The role of visualization to verify or adjust the hyper-parameters
  16. ...and 11 more sections

Key Result

Theorem 1

Given $N < n_z+1$ data pairs $\{{\boldsymbol{z}}_\ell,\boldsymbol{d}_\ell\}_{\ell=1}^N$, where $\{{\boldsymbol{z}}_\ell\}_{\ell=1}^N$ are linearly independent, there is a vector ${\boldsymbol{\uptheta}}^\star \in {\mathbb R}^{n_\uptheta}$ and a subspace $\Theta_{\text{interp}} \subset {\mathbb R}^{n

Figures (8)

  • Figure 1: Depiction of the HDSA-MD framework.
  • Figure 2: Value of \ref{['eqn:expectation_of_ratio']} for various smoothness hyper-parameters $\upbeta_{\boldsymbol{z}}$ and spatial dimensions $s=1,2,3$.
  • Figure 3: Depiction of the hyper-parameter specification process. The upper box indicates the algorithm-based initialization, which is performed once. The lower box indicates the visualization framework, which may be repeated multiple times if necessary to achieve appropriate hyper-parameters.
  • Figure 4: The panels from left to right correspond to: $\upalpha_{\boldsymbol{u}}, \upbeta_{\boldsymbol{u}}, \upalpha_{\boldsymbol{z}}$, and $\upbeta_{\boldsymbol{z}}$. The number of datapoints $N$ is varied on the horizontal axis and the range of hyper-parameters generated over all 50 datasets (generated uniquely for each $N$) is depicted by the vertical lines. The horizontal lines denote $\pm 20\%$ of the mean hyper-parameter value from all datasets.
  • Figure 5: First row: the discrepancy data $\{ \boldsymbol{d}_\ell \}_{\ell=1}^2$ (left) and the corresponding optimization variable data $\{ {\boldsymbol{z}}_\ell \}_{\ell=1}^2$ (center and right). The optimization variable inputs ${\boldsymbol{z}}^{(1)}$ and ${\boldsymbol{z}}^{(2)}$ are shown in the center and right columns, respectively. Second row: samples of ${\updelta}(\cdot,{\boldsymbol{\uptheta}}_i)$ evaluated at ${\boldsymbol{z}}_1$, ${\boldsymbol{z}}^{(1)}-{\boldsymbol{z}}_1$, and ${\boldsymbol{z}}^{(2)}-{\boldsymbol{z}}_1$, from left to right. Each curve is plotted in a gray scale with select samples plotted in color to highlight their characteristics. Third through sixth rows: analogous plots to the second row with each row corresponding to a different hyper-parameters divided by 10.
  • ...and 3 more figures

Theorems & Definitions (9)

  • Theorem 1
  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • proof
  • proof
  • proof
  • proof