Adaptive Gaussian Process Regression for Efficient Building of Surrogate Models in Inverse Problems

TL;DR

The paper tackles the challenge of solving many inverse problems where forward evaluations are costly by building an offline Gaussian process surrogate and using adaptive sequential design to jointly choose sample locations $p$ and evaluation tolerances under a finite budget. It develops an accuracy model that links surrogate error to parameter-reconstruction error, and a work model that ties evaluation cost to FE and Monte Carlo approximations, enabling a greedy, budget-aware design of experiments. The approach yields substantial speedups (often 100x–1000x over nonadaptive strategies) while maintaining reliable parameter estimates, as demonstrated on analytical and FEM-based examples; it also highlights practical limitations in estimator reliability for complex FE data and the sensitivity of derivative estimates to hyperparameters. Overall, the method provides a principled, data-driven framework for efficient surrogate-based parameter identification with adaptive resource allocation, offering a practical route for real-time or online inversion tasks.

Abstract

In a task where many similar inverse problems must be solved, evaluating costly simulations is impractical. Therefore, replacing the model $y$ with a surrogate model $y_s$ that can be evaluated quickly leads to a significant speedup. The approximation quality of the surrogate model depends strongly on the number, position, and accuracy of the sample points. With an additional finite computational budget, this leads to a problem of (computer) experimental design. In contrast to the selection of sample points, the trade-off between accuracy and effort has hardly been studied systematically. We therefore propose an adaptive algorithm to find an optimal design in terms of position and accuracy. Pursuing a sequential design by incrementally appending the computational budget leads to a convex and constrained optimization problem. As a surrogate, we construct a Gaussian process regression model. We measure the global approximation error in terms of its impact on the accuracy of the identified parameter and aim for a uniform absolute tolerance, assuming that $y_s$ is computed by finite element calculations. A priori error estimates and a coarse estimate of computational effort relate the expected improvement of the surrogate model error to computational effort, resulting in the most efficient combination of sample point and evaluation tolerance. We also allow for improving the accuracy of already existing sample points by continuing previously truncated finite element solution procedures.

Figures (10)

• Figure 1: Sketch of the design problem \ref{['eq:doe-incremental-reduced']} for $n=2$ candidate points. Level lines of the objective $E(v)$ are drawn by solid lines, whereas those of the constraints are indicated by dashed lines. Left: For $s>1$ there is a unique solution, which is likely not sparse. Middle: A smaller correlation length $L<1$ makes sparsity even less likely. Right: For $s<1$, the admissible sets are non-convex, and we may expect multiple local sparse minimizers.
• Figure 2: Adaptively added data points are indicated via black dots, with size indicating accuracy -- small points indicate low accuracy and vice versa. Red crosses are initial data points. The color mapping shows the isolines of the estimated local reconstruction error evaluated on a dense grid of $10^3$ points. This design was obtained using an incremental budget of $\Delta W = 10^{4}$. Left: Next to last design. The new point $p=(0.804, 0.241)$ to be added is marked with a green point and indicates the maximum value of the acquisition function. Right: Final parameter space after adaptive phase with $E_{\mathrm{ TOL }}(\mathcal{D})<\mathrm{TOL}$.
• Figure 3: Estimated global error $E(\mathcal{D})$ versus computational work. Left: Different amounts of incremental work $\Delta W$. Right: Different fixed evaluation accuracies $\epsilon$ compared with the curve for incremental work $\Delta W = 10$. Additionally we added two curves for a Latin Hypercube and a random sampling strategy.
• Figure 4: Left: Log-histogram of $\tilde{e}\cdot{e}^{-1}$. Right: Contour plot of $e_i$.
• Figure 5: Plot of the marginal distribution $f_{p_1}$ and $f_{p_2}$. Parameter reconstruction results for parameters $p_1$ and $p_2$ are shown by the red-blue dashed line. The artificial real parameters were chosen to be $p_{\textrm{real},1}=0.5$ and $p_{\textrm{real},2}=0.5$ and are shown by the red dashed line. The yellow dashed line indicates one standard deviation of the solution.

Theorems & Definitions (15)

• Remark
• Remark
• Remark
• Theorem 3.1
• proof
• Remark
• Corollary 3.1.1
• proof
• Corollary 3.1.2
• proof