Nonparametric Prior Learning in Differential Equation Modeling

TL;DR

The paper tackles Bayesian nonparametric inverse problems for PDEs by learning priors from historical tasks through a data driven prediction function. It extends PAC-Bayesian theory to infinite dimensional spaces and introduces data dependent priors via a differential privacy inspired assumption, yielding a tractable hyper posterior optimization. A MAP based learning algorithm is derived to update the hyper posterior from multiple tasks, with applicability to linear and nonlinear forward models including diffusion and Darcy flow. Numerical experiments on Darcy flow show that learned data dependent priors improve MAP accuracy, uncertainty quantification, and sampling efficiency, demonstrating practical benefits for complex PDE inverse problems.

Abstract

This paper addresses Bayesian inference related to partial differential equations (PDEs), particularly nonparametric regression constrained by PDEs. To effectively encode prior information, we propose a novel framework that learns a prediction function of the prior distribution from historical training datasets. We introduce hyper-prior and hyper-posterior distributions and derive a generalization error estimate, which accommodates data-dependent priors by extending the concept of differential privacy. Some mild conditions are given to validate the error estimate, where various typical PDEs such as diffusion and Darcy flow equations can be integrated. We thus formulate an infinite-dimensional optimization problem to obtain the point estimate of the hyper-posterior. Numerical examples demonstrate the performance of our proposed method in learning the prediction function of priors.

Figures (5)

• Figure 1: True function and estimated mean functions in the simple environment setting. (a) One of the functions in the testing dataset; (b) The learned mean function $f_m(S;\theta_1)$ of $\mathbb{P}_{S}^{\theta}$; (c) The learned mean function $f_m(\theta_1)$ of $\mathbb{P}^{\theta}$.
• Figure 2: Accumulated temperatures selected by effective sample size are shown in (a)-(c) for one test dataset from each case: the simple environment, the positive-valued branch, and the negative-valued branch of the complex environment. In all panels: blue dots represent unlearned Bayesian model temperatures, orange star dots denote learned data-independent Bayesian model temperatures, and green dots signify learned data-dependent Bayesian model temperatures.
• Figure 3: In all panels, red and blue dots denote the true function's projected coefficients and the base posterior mean's projected coefficients, respectively. Blue vertical lines mark $95\%$ credible intervals. Panels (a), (b), and (c) display results of simple environment under the priors $\mathbb{P}$, $\mathbb{P}^{\theta}$, and $\mathbb{P}_S^{\theta}$, respectively.
• Figure 4: True and estimated functions for Branches 1 and 2: (a) shows a random function of Branch 1; (b) and (c) present the mean functions of priors $\mathbb{P}_{S}^{\theta}$ and $\mathbb{P}^{\theta}$ for Branch 1, respectively; (d) depicts a random function of Branch 2; (e) and (f) display the corresponding mean functions from both priors for Branch 2.
• Figure 5: In all panels, red and blue dots denote the true function's projected coefficients and the base posterior mean's projected coefficients, respectively. Blue vertical lines mark $95\%$ credible intervals. Panels (a), (b), and (c) show results for a negative-valued branch (complex environment) using priors $\mathbb{P}$, $\mathbb{P}^{\theta}$, and $\mathbb{P}_S^{\theta}$, respectively.

Theorems & Definitions (15)

• Remark 2.1
• Remark 2.2
• Remark 2.3
• Theorem 2.4
• Remark 2.5
• Corollary 2.10
• Remark 2.11
• Remark 2.12
• Remark 2.13
• Theorem 3.3