Machine Learning for Partial Differential Equations

TL;DR

The paper analyzes how machine learning can transform the study and solution of partial differential equations (PDEs) by focusing on three core avenues: data-driven discovery of governing equations and coarse-grained closures, learning effective coordinates and reduced representations (including Koopman-based linearizations and reduced-order models), and learning solution operators to accelerate numerical PDE computation. It surveys methods such as PDE-FIND/SINDy for sparse discovery, Koopman theory and autoencoder-based ROMs for efficient representations, and neural operators (including DeepOnet) for mesh-free, discretization-invariant operator learning, along with practical acceleration techniques for solvers. The discussion highlights the importance of physics-informed learning, benchmarks, and human expertise, and identifies challenges like noise robustness, nonlocal effects, and data requirements, while outlining opportunities to discover new physics and extend multi-physics capabilities. Together, these approaches promise robust, interpretable, and scalable PDE modeling that integrates data-driven insights with traditional scientific computing.

Abstract

Partial differential equations (PDEs) are among the most universal and parsimonious descriptions of natural physical laws, capturing a rich variety of phenomenology and multi-scale physics in a compact and symbolic representation. This review will examine several promising avenues of PDE research that are being advanced by machine learning, including: 1) the discovery of new governing PDEs and coarse-grained approximations for complex natural and engineered systems, 2) learning effective coordinate systems and reduced-order models to make PDEs more amenable to analysis, and 3) representing solution operators and improving traditional numerical algorithms. In each of these fields, we summarize key advances, ongoing challenges, and opportunities for further development.

Figures (6)

• Figure 1: Sparse regression procedure to discover PDEs from data, demonstrated on the Navier--Stokes equations. A. Data is collected as snapshots of a solution to a PDE. B. Numerical derivatives are taken and data is compiled into a large matrix $\mathbf{\Theta}$, incorporating candidate terms for the PDE. C. Sparse regression is used to identify active terms in the PDE. D. Active terms in $\boldsymbol{\xi}$ are synthesized into a PDE. Modified from Rudy et al. Rudy2017sciadv.
• Figure 2: Illustration of sparse Bayesian PDE discovery applied to LES closure modeling in large-scale geophysical fluid simulations. Modified from Zanna and Bolton zanna2020data.
• Figure 3: A schematic of a neural network architecture used to discover a Koopman linearizing coordinate transformation. In this example, the nonlinear Burgers' equation is transformed into the linear heat equation.
• Figure 4: Zero-shot super-resolution: Vorticity field of the solution to the two-dimensional Navier-Stokes equation with viscosity $10^4$ ($Re=O(200)$); Ground truth on top and prediction on bottom. The model is trained on data that is discretized on a uniform 64 × 64 spatial grid and on a 20-point uniform temporal grid. The model is evaluated with a different initial condition that is discretized on a uniform 256 × 256 spatial grid and a 80-point uniform temporal grid ( From Kovachki et al kovachki2021neural ).
• Figure 5: Learning a reaction-diffusion with DeepOnet. (A) (left) An example of a random sample of the input function $u(x)$. (middle) The corresponding output function $s(x, t)$ at P different $(x, t)$ locations. (right) Pairing of inputs and outputs at the training data points. The total number of training data points is the product of $P$ times the number of samples of $u$. (B) Training error (blue) and test error (red) for different values of the number of random points $P$ when 100 random $u$ samples are used. (C) Training error (blue) and test error (red) for different number of u samples when $P = 100$. The shaded regions denote one-standard-derivation ( From Lu et al lu2019deeponet ).