Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Gaussian Process Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations

Carlos Mora, Amin Yousefpour, Shirin Hosseinmardi, Ramin Bostanabad

TL;DR

This work introduces NN-CoRes, a Gaussian-process–augmented framework for solving nonlinear PDEs that combines a deep neural network mean with a kernel-based GP prior. By employing kernel-weighted Corrective Residuals and a two-module training scheme, the method automatically enforces boundary and initial conditions while efficiently steering the solution to satisfy nonlinear PDE constraints, yielding robustness to initialization, architecture, and optimizer choices. The approach demonstrates superior performance across Burgers', nonlinear elliptic, Eikonal, and Navier–Stokes LDC problems, and extends to inverse problems by incorporating interior observations and unknown parameters. The findings indicate a practical, flexible pathway to leverage kernel methods within physics-informed learning, reducing sensitivity and potentially broadening applicability to multi-output and multiscale PDE systems.

Abstract

Physics-informed machine learning (PIML) has emerged as a promising alternative to conventional numerical methods for solving partial differential equations (PDEs). PIML models are increasingly built via deep neural networks (NNs) whose architecture and training process are designed such that the network satisfies the PDE system. While such PIML models have substantially advanced over the past few years, their performance is still very sensitive to the NN's architecture and loss function. Motivated by this limitation, we introduce kernel-weighted Corrective Residuals (CoRes) to integrate the strengths of kernel methods and deep NNs for solving nonlinear PDE systems. To achieve this integration, we design a modular and robust framework which consistently outperforms competing methods in solving a broad range of benchmark problems. This performance improvement has a theoretical justification and is particularly attractive since we simplify the training process while negligibly increasing the inference costs. Additionally, our studies on solving multiple PDEs indicate that kernel-weighted CoRes considerably decrease the sensitivity of NNs to factors such as random initialization, architecture type, and choice of optimizer. We believe our findings have the potential to spark a renewed interest in leveraging kernel methods for solving PDEs.

A Gaussian Process Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations

TL;DR

This work introduces NN-CoRes, a Gaussian-process–augmented framework for solving nonlinear PDEs that combines a deep neural network mean with a kernel-based GP prior. By employing kernel-weighted Corrective Residuals and a two-module training scheme, the method automatically enforces boundary and initial conditions while efficiently steering the solution to satisfy nonlinear PDE constraints, yielding robustness to initialization, architecture, and optimizer choices. The approach demonstrates superior performance across Burgers', nonlinear elliptic, Eikonal, and Navier–Stokes LDC problems, and extends to inverse problems by incorporating interior observations and unknown parameters. The findings indicate a practical, flexible pathway to leverage kernel methods within physics-informed learning, reducing sensitivity and potentially broadening applicability to multi-output and multiscale PDE systems.

Abstract

Physics-informed machine learning (PIML) has emerged as a promising alternative to conventional numerical methods for solving partial differential equations (PDEs). PIML models are increasingly built via deep neural networks (NNs) whose architecture and training process are designed such that the network satisfies the PDE system. While such PIML models have substantially advanced over the past few years, their performance is still very sensitive to the NN's architecture and loss function. Motivated by this limitation, we introduce kernel-weighted Corrective Residuals (CoRes) to integrate the strengths of kernel methods and deep NNs for solving nonlinear PDE systems. To achieve this integration, we design a modular and robust framework which consistently outperforms competing methods in solving a broad range of benchmark problems. This performance improvement has a theoretical justification and is particularly attractive since we simplify the training process while negligibly increasing the inference costs. Additionally, our studies on solving multiple PDEs indicate that kernel-weighted CoRes considerably decrease the sensitivity of NNs to factors such as random initialization, architecture type, and choice of optimizer. We believe our findings have the potential to spark a renewed interest in leveraging kernel methods for solving PDEs.
Paper Structure (29 sections, 2 theorems, 28 equations, 15 figures, 3 tables)

This paper contains 29 sections, 2 theorems, 28 equations, 15 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background on Physics-informed Machine Learning
  3. Outline of the Paper
  4. Neural Networks with Kernel-weighted
  5. Theoretical Rationale
  6. Proposed Framework
  7. Model Characteristics and Extensions
  8. Results and Discussions
  9. Description of the Benchmark Problems
  10. Implementation Details in Our Comparative Studies
  11. Architecture and Training
  12. Reference Solutions and Accuracy Metric
  13. Summary of Comparative Studies
  14. Sensitivity Analyses
  15. Loss Behavior
  16. ...and 14 more sections

Key Result

Theorem A3.1

The eigenfunctions of the real positive semidefinite kernel $c(\mathbf{x}, \mathbf{x}')$ whose eignenfunction expansion with respect to measure $\pi$ is $c(\mathbf{x}, \mathbf{x}') = \sum_{i=1}^N\alpha_i\psi_i(\mathbf{x})\psi_i(\mathbf{x}')$, are orthonormal. That is: where $\delta_{ij}$ denotes the Kronecker delta function. Following this theorem, we note that for a Hilbert space defined by the

Figures (15)

  • Figure 1: Flowchart of the proposed framework for solving the 1D Burgers' equation: We endow the solution $u(\mathbf{x})$ with a GP prior whose mean and covariance functions are parameterized via a deep NN and the Gaussian kernel in \ref{['eq: kernel-main']}, respectively. In module 1, we fix $\boldsymbol{\theta}$ to some random values and estimate the kernel parameters via MLE or heuristics such that the posterior GP conditioned on the BC/IC data faithfully reproduces $\textbf{u}$. Then, in Module 2, we estimate $\boldsymbol{\theta}$ by minimizing a loss function which only depends on the PDE since BC/IC are automatically satisfied.
  • Figure 2: Solving the 2D incompressible Navier-Stokes equations for the lid-driven cavity problem: With minor architectural changes on module two with respect to \ref{['fig: flowchart']}, our framework can also solve coupled PDE systems. Specifically, we endow each dependent variable with a GP prior. These GPs have independent kernels but a shared mean function that is parameterized via a deep neural network. Similar to \ref{['fig: flowchart']}, the loss function only depends on the PDE residuals and excludes data loss terms on BC/IC.
  • Figure 3: Model features: (a) In addition to improving the optimization of $\boldsymbol{\theta}$ in module two, kernel-weighted CoRes contribute to the model predictions and ensure strict satisfaction of the BCs. (b) Kernel-weighted CoRes automatically adapt to the domain geometry and are applicable to coupled PDE systems such as the Navier-Stokes equations. Here, we solve the unsteady LDC problem for $t \in \mathopen{}\left[0, 5\right]\mathclose{}$ and visualize the flow at $t=0.5$ and $t=5$.
  • Figure 4: Histograms of PDE loss gradients: NN-CoRes is in general more effective in minimizing its loss function as a larger portion of its gradients satisfy the first order optimality condition. While PINN$_\text{DW}$ has more near-zero gradients in the Eikonal problem, it does so at the expense of violating the BC loss term. All models in this figure have a $4\otimes20$ architecture.
  • Figure 5: Error analysis: NN-CoRes achieve smaller errors and increasing the size of their networks provides more improvement compared to methods such as PINN$_\text{DW}$.
  • ...and 10 more figures

Theorems & Definitions (4)

  • proof
  • Definition A3.1: Reproducing kernel Hilbert space
  • Theorem A3.1: Mercer's Theorem
  • Theorem A3.2: Representer Theorem