Multi-Fidelity Delayed Acceptance: hierarchical MCMC sampling for Bayesian inverse problems combining multiple solvers through deep neural networks

TL;DR

The paper tackles the computational bottleneck of Bayesian inverse problems for PDEs by introducing Multi-Fidelity Delayed Acceptance (MFDA), which offline-trains neural networks to fuse multiple low-fidelity solvers and correct their outputs toward high-fidelity predictions. Online, MFDA performs hierarchical, delayed-acceptance sampling using NN-corrected surrogates across fidelity levels, obviating the need for frequent high-fidelity evaluations. Across a groundwater transmissivity reconstruction and a nonlinear reaction–diffusion inverse problem, MFDA achieves posterior inferences with accuracy comparable to high-fidelity MH/MLDA methods while delivering substantial reductions in sampling time and overall cost. The results demonstrate improved mixing, longer coarse sub-chains, and a practical, scalable framework for PDE-based UQ with heterogeneous solvers and offline training capabilities.

Abstract

Inverse uncertainty quantification (UQ) tasks such as parameter estimation are computationally demanding whenever dealing with physics-based models, and typically require repeated evaluations of complex numerical solvers. When partial differential equations are involved, full-order models such as those based on the Finite Element Method can make traditional sampling approaches like Markov Chain Monte Carlo (MCMC) computationally infeasible. Although data-driven surrogate models may help reduce evaluation costs, their utility is often limited by the expense of generating high-fidelity data. In contrast, low-fidelity data can be produced more efficiently, although relying on them alone may degrade the accuracy of the inverse UQ solution. To address these challenges, we propose a Multi-Fidelity Delayed Acceptance scheme for Bayesian inverse problems. Extending the Multi-Level Delayed Acceptance framework, the method introduces multi-fidelity neural networks that combine the predictions of solvers of varying fidelity, with high fidelity evaluations restricted to an offline training stage. During the online phase, likelihood evaluations are obtained by evaluating the coarse solvers and passing their outputs to the trained neural networks, thereby avoiding additional high-fidelity simulations. This construction allows heterogeneous coarse solvers to be incorporated consistently within the hierarchy, providing greater flexibility than standard Multi-Level Delayed Acceptance. The proposed approach improves the approximation accuracy of the low fidelity solvers, leading to longer sub-chain lengths, better mixing, and accelerated posterior inference. The effectiveness of the strategy is demonstrated on two benchmark inverse problems involving (i) steady isotropic groundwater flow, (ii) an unsteady reaction-diffusion system, for which substantial computational savings are obtained.

Figures (16)

• Figure 1: Schematic representation of the MLDA scheme, for an instance of two low-fidelity solvers and one high-fidelity solver. At each level, a candidate $\boldsymbol{\theta}'$ is proposed by generating a sub-chain of length $J_{l-1}$. Each coarse level $l$ uses the surrogate $\mathbf{f}_\text{LF}^{(l)}$ and its corresponding likelihood $\tilde{\pi}_\text{LF}^{(l)}$, while the finest level uses $\mathbf{f}_\text{HF}$ and its corresponding likelihood ${\pi}$.
• Figure 2: Schematic representation of a multi-fidelity NN, for a generic instance of two low-fidelity solvers and a reference high-fidelity solver. Input parameters $\boldsymbol{\theta}$ are passed to the low-fidelity solvers. The outputs of the solvers and the input parameter $\boldsymbol{\theta}$ are passed to the first hidden layer of the NN. The subsequent hidden layers perform nonlinear transformations to capture complex correlations across fidelities. The final network output serves as an approximation of the high-fidelity model.
• Figure 3: Schematic representation of the Multi-Fidelity Delayed Acceptance (MFDA) framework for an instance of two low-fidelity solvers and one high-fidelity solver. First row: offline phase. A dataset of parameter samples is generated and each solver is evaluated for every parameter instance. Then multi-fidelity neural networks are trained to approximate the high-fidelity reference. Second row: online phase. A Markov chain of parameter samples is generated using a multi-level structure. At each level we evaluate the corresponding low-fidelity solvers and neural networks to compute the likelihood and acceptance rate.
• Figure 4: (a) Transmissivity field corresponding to mesh $\mathcal{T}_\mathrm{HF}$ and (b) hydraulic head using high-fidelity solver. Inverted triangles indicate sensor locations.
• Figure 5: Transmissivity fields corresponding to low-fidelity meshes: (a) $\mathcal{T}_1$, (b) $\mathcal{T}_2$, (c) $\mathcal{T}_3$, (d) $\mathcal{T}_4$.