Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equations

TL;DR

The paper presents numerical Gaussian processes (NGPs), a probabilistic framework where time-discretized PDEs induce covariance structures for Gaussian process priors, enabling uncertainty quantification from noisy initial data without spatial discretization. It develops two broad families of time-integration-informed priors: linear multistep methods and Runge-Kutta methods, each encoding the numerical scheme within the GP covariance, and demonstrates training, prediction, and uncertainty propagation across linear and nonlinear, time-dependent PDEs. Through Burgers', wave, advection, and heat equation benchmarks, the authors show accurate latent solutions and reliable uncertainty propagation over long time horizons, with convergence analyses highlighting first- and second-order temporal behavior depending on the scheme. The approach highlights the potential of probabilistic numerics for physics-guided learning, while noting computational costs and outlining future directions such as Kalman-style updates and model-discovery extensions.

Abstract

We introduce the concept of numerical Gaussian processes, which we define as Gaussian processes with covariance functions resulting from temporal discretization of time-dependent partial differential equations. Numerical Gaussian processes, by construction, are designed to deal with cases where: (1) all we observe are noisy data on black-box initial conditions, and (2) we are interested in quantifying the uncertainty associated with such noisy data in our solutions to time-dependent partial differential equations. Our method circumvents the need for spatial discretization of the differential operators by proper placement of Gaussian process priors. This is an attempt to construct structured and data-efficient learning machines, which are explicitly informed by the underlying physics that possibly generated the observed data. The effectiveness of the proposed approach is demonstrated through several benchmark problems involving linear and nonlinear time-dependent operators. In all examples, we are able to recover accurate approximations of the latent solutions, and consistently propagate uncertainty, even in cases involving very long time integration.

Figures (20)

• Figure 1: Burgers' equation: Initial data along with the posterior distribution of the solution at different time snapshots. The blue solid line represents the true data generating solution, while the dashed red line depicts the posterior mean. The shaded orange region illustrates the two standard deviations band around the mean. We are employing the backward Euler scheme with time step size $\Delta t = 0.01$. At each time step we generate $31$ artificial data points randomly located in the interval $[-1,1]$ according to a uniform distribution. These locations are highlighted by the ticks along the horizontal axis. Here, we set $\nu=0.01/\pi$ -- a value leading to the development of a non singular thin internal layer at $x=0$ that is notoriously hard to resolve by classical numerical methods basdevant1986spectral. (Code: http://bit.ly/2mnUiKT, Movie: http://bit.ly/2m1sKHw)
• Figure 2:
• Figure 3: Burgers' equation: Time evolution of the relative spatial $\mathcal{L}^2$-error up to the final integration time $T=1.0$. We are using the backward Euler scheme with a time step-size of $\Delta t = 0.01$, and the red dashed line illustrates the optimal first-order convergence rate. (Code: http://bit.ly/2mDY6It)
• Figure 4: Burgers' equation: Relative spatial $\mathcal{L}^2$-error versus step-size for the backward Euler scheme at time $T=0.1$. The number of noiseless initial and artificially generated data is set to be equal to $50$. (Code: http://bit.ly/2mDY6It)
• Figure 5: Burgers' equation: Relative spatial $\mathcal{L}^2$-error versus the number of noiseless initial as well as artificial data points used for the backward Euler scheme with time step-sizes of $\Delta t = 10^{-2}$ and $\Delta t = 10^{-3}$. (Code: http://bit.ly/2mDY6It)