Enhanced BPINN Training Convergence in Solving General and Multi-scale Elliptic PDEs with Noise

TL;DR

This work tackles uncertainty quantification for inverse PDEs when observations are noisy and the PDEs are multi-scale. It identifies convergence and step-size issues in traditional BPINN using $HMC$, especially on complex elliptic problems, and proposes MBPINN, a framework that reframes inference as SGD optimization and employs a Fourier-feature induced multi-scale DNN to capture both coarse and fine-scale behavior. The approach yields a robust, cost-efficient method that can handle general and multi-scale elliptic PDEs, outperforming $BPINN$ baselines in 1D and 2D experiments, and even failing gracefully where $HMC$-based methods struggle. The results suggest strong potential for physics-informed learning in ill-posed, noisy settings and multi-scale regimes, with practical impact on reliable parameter estimation and solution recovery.

Abstract

Bayesian Physics Informed Neural Networks (BPINN) have attracted considerable attention for inferring the system states and physical parameters of differential equations according to noisy observations. However, in practice, Hamiltonian Monte Carlo (HMC) used to estimate the internal parameters of the solver for BPINN often encounters these troubles including poor performance and awful convergence for a given step size used to adjust the momentum of those parameters. To address the convergence of HMC for the BPINN method and extend its application scope to multi-scale partial differential equations (PDE), we develop a robust multi-scale BPINN (dubbed MBPINN) method by integrating multi-scale deep neural networks (MscaleDNN) and the BPINN framework. In this newly proposed MBPINN method, we reframe HMC with Stochastic Gradient Descent (SGD) to ensure the most ``likely'' estimation is always provided, and we configure its solver as a Fourier feature mapping-induced MscaleDNN. This novel method offers several key advantages: (1) it is more robust than HMC, (2) it incurs less computational cost than HMC, and (3) it is more flexible for complex problems. We demonstrate the applicability and performance of the proposed method through some general Poisson and multi-scale elliptic problems in one and two-dimensional Euclidean spaces. Our findings indicate that the proposed method can avoid HMC failures and provide valid results. Additionally, our method is capable of handling complex elliptic PDE and producing comparable results for general elliptic PDE under the case of lower signal-to-noise rate. These findings suggest that our proposed approach has great potential for physics-informed machine learning for parameter estimation and solution recovery in the case of ill-posed problems.

Figures (14)

• Figure 1: The predictions of BPINN_HMC and exact solution for $\varepsilon=0.5$ and $\varepsilon=0.1$ when noise level is 0.01, respectively.
• Figure 2: The predictions of BPINN_HMC and exact solution for $\varepsilon=0.5$ when noise level is 0.05, respectively.
• Figure 3: The predictions of BPINN_HMC and exact solution for $\varepsilon=0.5$ when noise level 0.1, respectively.
• Figure 4: The decomposition of multi-scale function. Left: Original function, Middle: Coarse part, Right: Oscillation part
• Figure 5: The semantic diagram for MBPINN to solve multi-scale PDEs with noise