Randomized Physics-Informed Neural Networks for Bayesian Data Assimilation

TL;DR

This work targets uncertainty quantification in inverse PDE problems solved with PINNs, where the goal is to characterize the posterior over network parameters given noisy data. It introduces randomized PINN (rPINN), an optimization-based posterior approximation that injects Gaussian noise into the PINN loss and yields posterior samples by solving independent noisy minimizations; this approach is contrasted with BPINN methods based on Hamiltonian Monte Carlo (HMC) and Stein variational gradient descent (SVGD), as well as deep ensembles. Across linear and nonlinear Poisson equations and a diffusion problem with space-dependent coefficients, rPINN delivers informative predictive posteriors and substantial speedups (e.g., approximately 27x faster than HMC) while avoiding the convergence failures observed for HMC in nonlinear settings. The method leverages a weighted BPINN formulation and an empirical relation for the prior variance of the DNN weights, providing a robust, scalable tool for uncertainty quantification in inverse PDEs with potential applications in subsurface flow and other engineering contexts, where posterior sampling in high-dimensional neural nets is challenging.

Abstract

We propose a randomized physics-informed neural network (PINN) or rPINN method for uncertainty quantification in inverse partial differential equation (PDE) problems with noisy data. This method is used to quantify uncertainty in the inverse PDE PINN solutions. Recently, the Bayesian PINN (BPINN) method was proposed, where the posterior distribution of the PINN parameters was formulated using the Bayes' theorem and sampled using approximate inference methods such as the Hamiltonian Monte Carlo (HMC) and variational inference (VI) methods. In this work, we demonstrate that HMC fails to converge for non-linear inverse PDE problems. As an alternative to HMC, we sample the distribution by solving the stochastic optimization problem obtained by randomizing the PINN loss function. The effectiveness of the rPINN method is tested for linear and non-linear Poisson equations, and the diffusion equation with a high-dimensional space-dependent diffusion coefficient. The rPINN method provides informative distributions for all considered problems. For the linear Poisson equation, HMC and rPINN produce similar distributions, but rPINN is on average 27 times faster than HMC. For the non-linear Poison and diffusion equations, the HMC method fails to converge because a single HMC chain cannot sample multiple modes of the posterior distribution of the PINN parameters in a reasonable amount of time.

Figures (8)

• Figure 1: The comparison of using HMC, SVGD, and rPINN methods for sampling the posterior distribution of PINN parameters. The HMC and SVGD methods utilize the Bayesian PINN likelihood function and the neural network parameter prior (the blue dashed line). The rPINN method obtains posterior samples by solving the randomized (noise-perturbed) PINN minimization problem (the green dashed line). PDE derivatives are computed using automatic differentiation (AD).
• Figure 2: One-dimensional linear Poisson equation: posterior mean and confidence intervals of $u(x)$ and $f(x)$ computed from HMC (first column), rPINN (second column), SVGD (third column), and DE (fourth column) for (a) $\sigma = 0.1$ and, (b) $\sigma = 0.01$. The number of residual points is $N_f = 32$. The shaded areas represent $95\%$ confidence intervals (the pointwise mean plus/minus two standard deviations). The HMC chains produce fully overlapping confidence intervals. Therefore, the results for one chain can be seen on the HMC plots.
• Figure 3: One-dimensional non-linear Poisson problem: posterior predictions of $u(x)$ and $f(x)$ given by HMC (first column), rPINN (second column), SVGD (third column), and DE (fourth column) for (a) $\sigma = 0.1$ and (b) $\sigma = 0.01$. The number of residual points is $N_f = 32$.
• Figure 4: Two-dimensional diffusion equation: (a) the reference $y_{\text{ref}}(\mathbf{x})=\ln k_{\text{ref}}(\mathbf{x})$ field with a correlation length $\lambda=0.5$ and (b) the reference hydraulic head field $h_{\text{ref}}(\mathbf{x})$.
• Figure 5: Two-dimensional diffusion equation: rPINN predictions of $y(\mathbf{x})$ for (a) $\sigma = 1$, (b) $\sigma = 0.1$, and (c) $\sigma = 0.01$: (first column) the posterior mean estimates; (second column) point errors in the predicted mean $y$ with respect to the reference $y$ field; (third column) pointwise posterior standard deviation; and (fourth column) the coverage of the reference field with the $95\%$ credibility interval, where the zero and one values correspond to the reference field being outside and inside the credibility interval, respectively.