Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hierarchical Gaussian Random Fields for Multilevel Markov Chain Monte Carlo: Coupling Stochastic Partial Differential Equation and The Karhunen-Loève Decomposition

Sohail Reddy

TL;DR

This paper develops a hierarchical, structure-preserving framework for sampling Gaussian random fields (GRFs) in high-dimensional Bayesian inference by coupling Karhunen–Loève (KL) expansions with stochastic PDE (SPDE) sampling across multilevel grids. It introduces covariance-preserving multilevel projections and a coupled KL–multigrid decomposition to ensure ergodicity and accurate posterior sampling while reducing the effective dimensionality of the sample space. The authors integrate this framework into a Multilevel MCMC (MLMCMC) with delayed acceptance and demonstrate its effectiveness on a Bayesian subsurface Darcy flow problem, showing that an optimal coarse-subspace (e.g., around 50 KL modes) balances accuracy and computational cost. The results indicate substantial improvements in sampling efficiency and posterior accuracy over traditional SPDE-only approaches, highlighting the potential for scalable uncertainty quantification in high-dimensional PDE-based inverse problems. The work also outlines future directions for rigorously identifying the optimal coarse subspace and applying machine learning to accelerate forward-model evaluations at multiple levels.

Abstract

This work introduces structure preserving hierarchical decompositions for sampling Gaussian random fields (GRF) within the context of multilevel Bayesian inference in high-dimensional space. Existing scalable hierarchical sampling methods, such as those based on stochastic partial differential equation (SPDE), often reduce the dimensionality of the sample space at the cost of accuracy of inference. Other approaches, such that those based on Karhunen-Loève (KL) expansions, offer sample space dimensionality reduction but sacrifice GRF representation accuracy and ergodicity of the Markov Chain Monte Carlo (MCMC) sampler, and are computationally expensive for high-dimensional problems. The proposed method integrates the dimensionality reduction capabilities of KL expansions with the scalability of stochastic partial differential equation (SPDE)-based sampling, thereby providing a robust, unified framework for high-dimensional uncertainty quantification (UQ) that is scalable, accurate, preserves ergodicity, and offers dimensionality reduction of the sample space. The hierarchy in our multilevel algorithm is derived from the geometric multigrid hierarchy. By constructing a hierarchical decomposition that maintains the covariance structure across the levels in the hierarchy, the approach enables efficient coarse-to-fine sampling while ensuring that all samples are drawn from the desired distribution. The effectiveness of the proposed method is demonstrated on a benchmark subsurface flow problem, demonstrating its effectiveness in improving computational efficiency and statistical accuracy. Our proposed technique is more efficient, accurate, and displays better convergence properties than existing methods for high-dimensional Bayesian inference problems.

Hierarchical Gaussian Random Fields for Multilevel Markov Chain Monte Carlo: Coupling Stochastic Partial Differential Equation and The Karhunen-Loève Decomposition

TL;DR

This paper develops a hierarchical, structure-preserving framework for sampling Gaussian random fields (GRFs) in high-dimensional Bayesian inference by coupling Karhunen–Loève (KL) expansions with stochastic PDE (SPDE) sampling across multilevel grids. It introduces covariance-preserving multilevel projections and a coupled KL–multigrid decomposition to ensure ergodicity and accurate posterior sampling while reducing the effective dimensionality of the sample space. The authors integrate this framework into a Multilevel MCMC (MLMCMC) with delayed acceptance and demonstrate its effectiveness on a Bayesian subsurface Darcy flow problem, showing that an optimal coarse-subspace (e.g., around 50 KL modes) balances accuracy and computational cost. The results indicate substantial improvements in sampling efficiency and posterior accuracy over traditional SPDE-only approaches, highlighting the potential for scalable uncertainty quantification in high-dimensional PDE-based inverse problems. The work also outlines future directions for rigorously identifying the optimal coarse subspace and applying machine learning to accelerate forward-model evaluations at multiple levels.

Abstract

This work introduces structure preserving hierarchical decompositions for sampling Gaussian random fields (GRF) within the context of multilevel Bayesian inference in high-dimensional space. Existing scalable hierarchical sampling methods, such as those based on stochastic partial differential equation (SPDE), often reduce the dimensionality of the sample space at the cost of accuracy of inference. Other approaches, such that those based on Karhunen-Loève (KL) expansions, offer sample space dimensionality reduction but sacrifice GRF representation accuracy and ergodicity of the Markov Chain Monte Carlo (MCMC) sampler, and are computationally expensive for high-dimensional problems. The proposed method integrates the dimensionality reduction capabilities of KL expansions with the scalability of stochastic partial differential equation (SPDE)-based sampling, thereby providing a robust, unified framework for high-dimensional uncertainty quantification (UQ) that is scalable, accurate, preserves ergodicity, and offers dimensionality reduction of the sample space. The hierarchy in our multilevel algorithm is derived from the geometric multigrid hierarchy. By constructing a hierarchical decomposition that maintains the covariance structure across the levels in the hierarchy, the approach enables efficient coarse-to-fine sampling while ensuring that all samples are drawn from the desired distribution. The effectiveness of the proposed method is demonstrated on a benchmark subsurface flow problem, demonstrating its effectiveness in improving computational efficiency and statistical accuracy. Our proposed technique is more efficient, accurate, and displays better convergence properties than existing methods for high-dimensional Bayesian inference problems.

Paper Structure

This paper contains 13 sections, 2 theorems, 57 equations, 6 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Sampling Gaussian Random Fields
  3. Sampling Technique: Karhunen–Loève Expansion
  4. Sampling Technique: Stochastic PDE Approach
  5. Multilevel Decomposition for Hierarchical Sampling
  6. Hierarchical Sampling: Multigrid Decomposition
  7. Hierarchical Sampling: Coupled Karhunen–Loève -Multigrid Decomposition
  8. Bayesian Inference: Multi-Level Markov Chain Monte Carlo
  9. Subsurface Flow: Darcy's Equations
  10. Results
  11. Problem Formulation
  12. Multilevel Bayesian Inference in Subsurface Flows
  13. Conclusion

Key Result

Lemma 1

Let $\eta_\ell \sim \mathcal{N}(0,K_\ell)$ be a random function on level $\ell$ with covariance $K_\ell$, and $\eta_\ell^L = \mathcal{I}\eta_\ell$ is the realization of $\eta_\ell$ in the basis of level $L$ with $\mathcal{R} = \mathcal{I}^T$. Then, $\mathbb{E}_{} \left[ \mathcal{Q}_L\eta_\ell^L~(\ma

Figures (6)

  • Figure 1: A realization of a Gaussian random field across the levels of a two-level hierarchy sampled using the SPDE approach with: a) a sample on the coarse level, b) the coarse sample on the fine-level, c) fine-level sample in the coarse complement, and d) the complete multilevel realization of the GRF.
  • Figure 2: A realization of a Gaussian random field across the levels of a two-level hierarchy sampled using the Karhunen–Loève and SPDE sampler on the coarse and fine level, respectively. on the approach with: (a,d,g): a sample on the coarse level, (b,e,h): a fine-level sample in the space complement to $\widehat{\Theta}_{\mathpzc{m}} = \mathrm{span}(\widehat{\Psi})$, and (c,f,i): the complete multilevel realization of the GRF. $\widehat{\Theta}_{\mathpzc{m}}$ represents the use of $\mathpzc{m}$ modes on the coarse level (i.e. $\mathpzc{m} = |\widehat{\Psi}|$).
  • Figure 3: The reference/observational (a) Gaussian random field and (b) the resulting pressure field.
  • Figure 4: Realizations of GRF samples on different levels and corresponding pressure fields obtained via multilevel using SPDE (top), KL with 10 modes (middle) and KL with 50 modes (bottom).
  • Figure 5: Autocorrelation function of $Y$ and $Q$ on different levels obtained using: a) SPDE sampling, and KL-SPDE sampling with: b) 10 modes, c) 50 modes and d) 75 modes.
  • ...and 1 more figures

Theorems & Definitions (6)

  • Lemma 1
  • proof
  • Theorem 1
  • proof
  • Remark 1
  • Remark 2