Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accurate, scalable, and efficient Bayesian optimal experimental design with derivative-informed neural operators

Jinwoo Go, Peng Chen

TL;DR

An accurate, scalable, and efficient computational framework based on derivative-informed neural operators (DINO) to reduce the parameter dimensions and it is demonstrated that the proposed computational framework is scalable to preserve the accuracy for increasing parameter dimensions and achieves high computational efficiency.

Abstract

We consider optimal experimental design (OED) problems in selecting the most informative observation sensors to estimate model parameters in a Bayesian framework. Such problems are computationally prohibitive when the parameter-to-observable (PtO) map is expensive to evaluate, the parameters are high-dimensional, and the optimization for sensor selection is combinatorial and high-dimensional. To address these challenges, we develop an accurate, scalable, and efficient computational framework based on derivative-informed neural operators (DINO). We propose to use derivative-informed dimension reduction to reduce the parameter dimensions, based on which we train DINO with derivative information as an accurate and efficient surrogate for the PtO map and its derivative. Moreover, we derive DINO-enabled efficient formulations in computing the maximum a posteriori (MAP) point, the eigenvalues of approximate posterior covariance, and three commonly used optimality criteria for the OED problems. Furthermore, we provide detailed error analysis for the approximations of the MAP point, the eigenvalues, and the optimality criteria. We also propose a modified swapping greedy algorithm for the sensor selection optimization and demonstrate that the proposed computational framework is scalable to preserve the accuracy for increasing parameter dimensions and achieves high computational efficiency, with an over 1000$\times$ speedup accounting for both offline construction and online evaluation costs, compared to high-fidelity Bayesian OED solutions for a three-dimensional nonlinear convection-diffusion-reaction example with tens of thousands of parameters.

Accurate, scalable, and efficient Bayesian optimal experimental design with derivative-informed neural operators

TL;DR

An accurate, scalable, and efficient computational framework based on derivative-informed neural operators (DINO) to reduce the parameter dimensions and it is demonstrated that the proposed computational framework is scalable to preserve the accuracy for increasing parameter dimensions and achieves high computational efficiency.

Abstract

We consider optimal experimental design (OED) problems in selecting the most informative observation sensors to estimate model parameters in a Bayesian framework. Such problems are computationally prohibitive when the parameter-to-observable (PtO) map is expensive to evaluate, the parameters are high-dimensional, and the optimization for sensor selection is combinatorial and high-dimensional. To address these challenges, we develop an accurate, scalable, and efficient computational framework based on derivative-informed neural operators (DINO). We propose to use derivative-informed dimension reduction to reduce the parameter dimensions, based on which we train DINO with derivative information as an accurate and efficient surrogate for the PtO map and its derivative. Moreover, we derive DINO-enabled efficient formulations in computing the maximum a posteriori (MAP) point, the eigenvalues of approximate posterior covariance, and three commonly used optimality criteria for the OED problems. Furthermore, we provide detailed error analysis for the approximations of the MAP point, the eigenvalues, and the optimality criteria. We also propose a modified swapping greedy algorithm for the sensor selection optimization and demonstrate that the proposed computational framework is scalable to preserve the accuracy for increasing parameter dimensions and achieves high computational efficiency, with an over 1000 speedup accounting for both offline construction and online evaluation costs, compared to high-fidelity Bayesian OED solutions for a three-dimensional nonlinear convection-diffusion-reaction example with tens of thousands of parameters.
Paper Structure (31 sections, 3 theorems, 91 equations, 18 figures, 6 tables, 1 algorithm)

This paper contains 31 sections, 3 theorems, 91 equations, 18 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem statement
  3. Bayesian inverse problem
  4. Bayesian optimal experimental design
  5. High-fidelity approximations
  6. High-fidelity discretization
  7. Laplace and low-rank approximations
  8. High-fidelity Bayesian OED
  9. Computational challenges
  10. Bayesian OED with DINO
  11. Derivative-informed dimension reduction
  12. Derivative-informed neural operators
  13. Efficient computation of the MAP point and the eigenpairs with DINO
  14. Swapping greedy algorithm
  15. Computational complexity
  16. ...and 16 more sections

Key Result

Theorem 1

Under Assumption ass:assumptions, if $||{\boldsymbol{\eta}}^* - \hat{{\boldsymbol{\eta}}}^*|| \leq R_{\boldsymbol{\eta}}$, we have the following error bound for the approximation of high-fidelity MAP point by the surrogate MAP point where $c_1 > 0, c_2>0,$ and $c_3 > 0$ are positive constants independent of $\varepsilon_1, \varepsilon_2,$ and $\varepsilon_3$.

Figures (18)

  • Figure 1: A random prior sample $m \sim {\mathcal{N}}(0, {\mathcal{C}})$ (left), the high-fidelity solution $u(m)$ by finite element approximation (middle), and the observation data ${\boldsymbol{y}}$ at all the candidate observation points (right). Top: diffusion problem. Bottom: CDR problem.
  • Figure 2: Decay of the eigenvalues of the prior covariance $\Gamma_\text{prior}$ used in the KLE approximation of the parameter \ref{['eq:kle']} and the eigenvalues of the generalized eigenvalue problem \ref{['eq:gIAS']} used in DIS approximation of the parameter \ref{['eq:ias']}. Left: for the diffusion problem. Right: for the CDR problem.
  • Figure 3: KLE bases 1, 4, 16, 64 (top). DIS bases 1, 4, 16, 64 for the diffusion (middle) and CDR (bottom) problems.
  • Figure 4: Relative errors of the input projection (top and middle rows for the two 2D examples, respectively) and output projection (bottom row for the CDR problem) for the observables (left), the MAP point (middle), and the Jacobian (right). We use DIS \ref{['eq:ias']} and KLE \ref{['eq:kle']} basis for input projection, and DOS \ref{['eq:oas']} and PCA \ref{['eq:pca']} basis for output projection.
  • Figure 5: Mean of relative errors with increasing training size for the neural network approximations of the PtO map $F$, the reduced Jacobian ${J_r}$, and the MAP point ${\boldsymbol{m}}_\text{MAP}$. The solid and dashed line results are for the neural networks trained with and without the Jacobian. Diffusion (left) and CDR (right) problems.
  • ...and 13 more figures

Theorems & Definitions (8)

  • Remark 1
  • Remark 2
  • Theorem 1
  • proof
  • Theorem 2
  • proof
  • Theorem 3
  • proof