Posterior sampling with Adaptive Gaussian Processes in Bayesian parameter identification

TL;DR

A fully adaptive greedy approach to posterior sampling by Monte Carlo methods using Gaussian process regression as surrogate, which shows a significant reduction of the computational effort compared to just position-adaptive and static designs.

Abstract

Posterior sampling by Monte Carlo methods provides a more comprehensive solution approach to inverse problems than computing point estimates such as the maximum posterior using optimization methods, at the expense of usually requiring many more evaluations of the forward model. Replacing computationally expensive forward models by fast surrogate models is an attractive option. However, computing the simulated training data for building a sufficiently accurate surrogate model can be computationally expensive in itself, leading to the design of computer experiments problem of finding evaluation points and accuracies such that the highest accuracy is obtained given a fixed computational budget. Here, we consider a fully adaptive greedy approach to this problem. Using Gaussian process regression as surrogate, samples are drawn from the available posterior approximation while designs are incrementally defined by solving a sequence of optimization problems for evaluation accuracy and positions. The selection of training designs is tailored towards representing the posterior to be sampled as good as possible, while the interleaved sampling steps discard old inaccurate samples in favor of new, more accurate ones. Numerical results show a significant reduction of the computational effort compared to just position-adaptive and static designs.

Figures (10)

• Figure 1: Surrogate mean error and training points (black) for different steps of a realization of \ref{['algo:AGP']} under the experimental conditions described in Experiment \ref{['exp:2d']}. The size of the each training point is proportional to the computational work spent in the point.
• Figure 2: Ground truth forward model for the 2-d parameter space.
• Figure 3: Accumulated samples (black line), drawn samples (green bar), discarded samples (red bar) for the 2-d example.
• Figure 4: Distribution of the logarithm of the evaluation tolerances for AGP-geom with target KL divergence and different simulation costs. The logarithm of the default tolerance is represented by the black vertical line. On average 34 points are included in the training set, resulting in around 850 different evaluation accuracies for each of the costs. The density function is reconstructed by scipy.stats.gaussian_kde.
• Figure 5: Convergence rates for the two different error models and four different FE simulation costs on a 2-d parameter space. The thinner lines represent individual runs, the intermediate-thickness lines the average over 5 runs with the same measurements and posterior and the thicker lines the average on the whole 25 runs.

Theorems & Definitions (13)

• Remark 1
• Theorem 1
• proof
• Lemma 1
• proof
• Remark 2
• Corollary 1
• proof
• Corollary 2
• proof