PDE-DKL: PDE-constrained deep kernel learning in high dimensionality

TL;DR

This work tackles solving high-dimensional PDEs under data scarcity by proposing PDE-DKL, a hybrid framework that uses deep kernel learning to map high-dimensional PDE inputs into a low-dimensional latent space, while enforcing PDE constraints through Gaussian process regression. The PDE constraint is propagated through linear operators, ensuring that the latent GP and the reconstructed forcing term $f=\mathcal{A}[u]$ satisfy the governing equations, with uncertainty quantified via GP posteriors. PDE-DKL extends to deep kernels by embedding a neural network into an ARD kernel, enabling non-stationary, expressive representations that remain scalable in high dimensions. The approach is validated on four benchmark PDEs up to 50 dimensions, demonstrating improved accuracy and uncertainty quantification over PDE-GP, and showing practical feasibility where traditional GP methods struggle. Overall, PDE-DKL provides a principled, scalable pathway for physics-informed, uncertainty-aware surrogate modeling of complex, high-dimensional PDEs.

Abstract

Many physics-informed machine learning methods for PDE-based problems rely on Gaussian processes (GPs) or neural networks (NNs). However, both face limitations when data are scarce and the dimensionality is high. Although GPs are known for their robust uncertainty quantification in low-dimensional settings, their computational complexity becomes prohibitive as the dimensionality increases. In contrast, while conventional NNs can accommodate high-dimensional input, they often require extensive training data and do not offer uncertainty quantification. To address these challenges, we propose a PDE-constrained Deep Kernel Learning (PDE-DKL) framework that combines DL and GPs under explicit PDE constraints. Specifically, NNs learn a low-dimensional latent representation of the high-dimensional PDE problem, reducing the complexity of the problem. GPs then perform kernel regression subject to the governing PDEs, ensuring accurate solutions and principled uncertainty quantification, even when available data are limited. This synergy unifies the strengths of both NNs and GPs, yielding high accuracy, robust uncertainty estimates, and computational efficiency for high-dimensional PDEs. Numerical experiments demonstrate that PDE-DKL achieves high accuracy with reduced data requirements. They highlight its potential as a practical, reliable, and scalable solver for complex PDE-based applications in science and engineering.

Figures (9)

• Figure 1: Conceptual diagram of PDE-constrained deep kernel learning (PDE-DKL): The initialisation training involves using PDE coordinates $\vb*{q}$ as inputs to a deep kernel. This kernel is formed by integrating an NN with parameters $\vb*{\omega}$ into a base kernel with hyperparameters $\vb*{\theta}$. It serves as the covariance function for the latent GP solution $u$. Applying a linear operator $\mathcal{A}$ to the latent GP produces a GP for the forcing term $f$. Subsequently, the joint kernel $\mathbf{K}_{\texttt{DKL}}$ is employed to minimize the negative log marginal likelihood $\mathcal{L}_{\mathcal{N L M L}}$, determining the parameters $\left\{\vb*{\theta}, \vb*{\omega}, \sigma_u^2, \sigma_f^2\right\}$. The trained model evaluates the solution field $u^*(\boldsymbol{q})$ in the prediction stage.
• Figure 2: Comparison of PDE-GP (black dots) and PDE-DKL (blue dots) in predicting the solution term $u$ and reconstructing the forcing term $f$. Subplots ($\textbf{a}$) and (b) show the predictions for $u$ using GP and DKL, respectively, while subplots (c) and (d) show the reconstructions for $f$ using GP and DKL respectively.
• Figure 3: Comparison of PDE-GP and PDE-DKL methods in solving the ten-dimensional Poisson equation. Subplots (a) and (b) show the predicted solutions for $u$ using PDE-GP and PDE-DKL, respectively. Subplots (c) and (d) display the reconstructed source term $f$ using PDE-GP and PDE-DKL, respectively.
• Figure 4: Accuracy of PDE-DKL and PDE-GP in solving high-dimensional Poisson equations. Subfigure ($\textbf{a}$) details the accuracy and uncertainty of PDE-DKL in solving for variable $u$ in a 50-dimensional Poisson equation. Subfigure ($\textbf{b}$) illustrates the accuracy and uncertainty in reconstructing $f$ using PDE-DKL, demonstrating its superior performance and efficiency in handling large-scale computations.
• Figure 5: Subfigures (a) and (b) show scatter plots of predicted versus true values of $u$ for PDE-GP and PDE-DKL, respectively. Subplots (c) and (d) display the reconstructed source term $f$ using PDE-GP and PDE-DKL, respectively.