Solving High Frequency and Multi-Scale PDEs with Gaussian Processes

TL;DR

This work tackles the difficulty of solving high-frequency and multi-scale PDEs with neural approaches by introducing GP-HM, a Gaussian-process solver that models the solution's power spectrum with a mixture of Student-$t$ (or Gaussian) components. The inverse Fourier transform yields a real-valued covariance that encodes target frequencies, with component weights learned in the log domain, effectively implementing a Jeffreys prior to prune irrelevant frequencies. By placing collocation points on a grid and using a product covariance, the method achieves a Kronecker-structured, scalable GP formulation capable of computing PDE derivatives and enforcing boundary conditions without low-rank approximations. Empirical results on Poisson, Allen-Cahn, and advection benchmarks show competitive or superior accuracy (relative $L_2$ errors around $10^{-3}$ to $10^{-7}$) and learned frequencies that align with ground truth, highlighting the practical potential of frequency-aware GP solvers for complex PDEs.

Abstract

Machine learning based solvers have garnered much attention in physical simulation and scientific computing, with a prominent example, physics-informed neural networks (PINNs). However, PINNs often struggle to solve high-frequency and multi-scale PDEs, which can be due to spectral bias during neural network training. To address this problem, we resort to the Gaussian process (GP) framework. To flexibly capture the dominant frequencies, we model the power spectrum of the PDE solution with a student $t$ mixture or Gaussian mixture. We apply the inverse Fourier transform to obtain the covariance function (by Wiener-Khinchin theorem). The covariance derived from the Gaussian mixture spectrum corresponds to the known spectral mixture kernel. Next, we estimate the mixture weights in the log domain, which we show is equivalent to placing a Jeffreys prior. It automatically induces sparsity, prunes excessive frequencies, and adjusts the remaining toward the ground truth. Third, to enable efficient and scalable computation on massive collocation points, which are critical to capture high frequencies, we place the collocation points on a grid, and multiply our covariance function at each input dimension. We use the GP conditional mean to predict the solution and its derivatives so as to fit the boundary condition and the equation itself. As a result, we can derive a Kronecker product structure in the covariance matrix. We use Kronecker product properties and multilinear algebra to promote computational efficiency and scalability, without low-rank approximations. We show the advantage of our method in systematic experiments. The code is released at \url{https://github.com/xuangu-fang/Gaussian-Process-Slover-for-High-Freq-PDE}.

Figures (9)

• Figure 1: Prediction for the 1D Poisson equation with solution $\sin(x) + 0.1\sin(20x) + 0.05\cos(100x)$.
• Figure 2: Prediction for the 1D Poisson equation with solution $\sin(500x)-2(x-0.5)^2$.
• Figure 3: Point-wise solution error for 2D Allen-cahn equation, and the solution is $\left(\sin(x) + 0.1\sin(20x) + \cos(100x)\right) \left(\sin(y) + 0.1\sin(20y) + \cos(100y)\right)$.
• Figure 4: The learned component weights and frequency values. For each number pair a(b) in the figure, "a" is the learned frequency by GP-HM, and "b" is the ground-truth. The expressions on the top are the solutions.
• Figure 5: Prediction for the 1D Poisson equation with solution $x\sin(200x)$.