Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tensor Gaussian Processes: Efficient Solvers for Nonlinear PDEs

Qiwei Yuan, Zhitong Xu, Yinghao Chen, Yiming Xu, Houman Owhadi, Shandian Zhe

TL;DR

TGPS is a tensor-GP-based solver that models factor functions along each input dimension using one-dimensional GPs and combines them via tensor decomposition to approximate the full solution, substantially lowering computational complexity, and enabling scalability to massive collocation sets.

Abstract

Machine learning solvers for partial differential equations (PDEs) have attracted growing interest. However, most existing approaches, such as neural network solvers, rely on stochastic training, which is inefficient and typically requires a great many training epochs. Gaussian process (GP)/kernel-based solvers, while mathematical principled, suffer from scalability issues when handling large numbers of collocation points often needed for challenging or higher-dimensional PDEs. To overcome these limitations, we propose TGPS, a tensor-GP-based solver that models factor functions along each input dimension using one-dimensional GPs and combines them via tensor decomposition to approximate the full solution. This design reduces the task to learning a collection of one-dimensional GPs, substantially lowering computational complexity, and enabling scalability to massive collocation sets. For efficient nonlinear PDE solving, we use a partial freezing strategy and Newton's method to linerize the nonlinear terms. We then develop an alternating least squares (ALS) approach that admits closed-form updates, thereby substantially enhancing the training efficiency. We establish theoretical guarantees on the expressivity of our model, together with convergence proof and error analysis under standard regularity assumptions. Experiments on several benchmark PDEs demonstrate that our method achieves superior accuracy and efficiency compared to existing approaches.

Tensor Gaussian Processes: Efficient Solvers for Nonlinear PDEs

TL;DR

TGPS is a tensor-GP-based solver that models factor functions along each input dimension using one-dimensional GPs and combines them via tensor decomposition to approximate the full solution, substantially lowering computational complexity, and enabling scalability to massive collocation sets.

Abstract

Machine learning solvers for partial differential equations (PDEs) have attracted growing interest. However, most existing approaches, such as neural network solvers, rely on stochastic training, which is inefficient and typically requires a great many training epochs. Gaussian process (GP)/kernel-based solvers, while mathematical principled, suffer from scalability issues when handling large numbers of collocation points often needed for challenging or higher-dimensional PDEs. To overcome these limitations, we propose TGPS, a tensor-GP-based solver that models factor functions along each input dimension using one-dimensional GPs and combines them via tensor decomposition to approximate the full solution. This design reduces the task to learning a collection of one-dimensional GPs, substantially lowering computational complexity, and enabling scalability to massive collocation sets. For efficient nonlinear PDE solving, we use a partial freezing strategy and Newton's method to linerize the nonlinear terms. We then develop an alternating least squares (ALS) approach that admits closed-form updates, thereby substantially enhancing the training efficiency. We establish theoretical guarantees on the expressivity of our model, together with convergence proof and error analysis under standard regularity assumptions. Experiments on several benchmark PDEs demonstrate that our method achieves superior accuracy and efficiency compared to existing approaches.
Paper Structure (22 sections, 5 theorems, 75 equations, 6 figures, 8 tables)

This paper contains 22 sections, 5 theorems, 75 equations, 6 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Our Method
  4. Model
  5. Algorithm
  6. Theoretical Analysis
  7. Related Work
  8. Numerical Experiments
  9. Solution Accuracy
  10. Running Efficiency
  11. Conclusion
  12. PDE Benchmarks
  13. Method Details
  14. Proof of Lemma \ref{['thm:1']} and \ref{['thm:2']}
  15. Bound of RKHS Norm
  16. ...and 7 more sections

Key Result

Lemma 4.1

Suppose that $u^* \in H^k(\Omega)$ for some $k\in\mathbb N$, where $H^k(\Omega)$ denotes the Sobolev space consisting of functions whose weak derivatives up to the order of $k$ have finite $L^2$ norms. Given an arbitrary error $0<\varepsilon<1$, if we model $u$ as the CP format in eq:cp and let $\ov Moreover, if we further assume $u^*\in H_{{\bf v}}^k(\Omega)$ for some ${\bf v} \in \mathbb{R}_+^D$

Figures (6)

  • Figure 1: Relative $L^2$ error vs. the number of collocation points in solving higher-dimensional PDEs.
  • Figure 2: Training curves: Training loss vs. Number of iterations.
  • Figure 3: Running time (in seconds) vs. Relative $L^2$ error.
  • Figure 4: Point-wise error. Inside each parenthesis is the number of collocation points.
  • Figure 5: Solving the nonlinear elliptic PDE on irregular domains. The first column shows the ground-truth while the remaining columns the point-wise error of each method.
  • ...and 1 more figures

Theorems & Definitions (9)

  • Lemma 4.1
  • Lemma 4.2
  • Lemma 4.4
  • Proposition 4.5
  • Definition C.1: Sobolev Space and Weighed Sobolev Space
  • proof
  • Lemma D.1
  • proof
  • proof