PDE-Net: Learning PDEs from Data

TL;DR

Proposes PDE-Net, a deep feed-forward framework that learns both differential operators via constrained convolutional filters and the nonlinear response F via a pointwise network to discover and predict PDE-driven dynamics from data. The method leverages sum-rule–based constraints and moment matrices to tie learned filters to specific derivative orders, enabling identifiability of the governing equations. Through linear convection-diffusion and diffusion with nonlinear sources, PDE-Net demonstrates long-time predictive accuracy and the ability to recover underlying PDE structure, with performance improved by deeper architectures and larger filters. This approach offers a transparent, data-driven pathway to uncovering governing PDEs in complex systems while maintaining strong predictive power.

Abstract

In this paper, we present an initial attempt to learn evolution PDEs from data. Inspired by the latest development of neural network designs in deep learning, we propose a new feed-forward deep network, called PDE-Net, to fulfill two objectives at the same time: to accurately predict dynamics of complex systems and to uncover the underlying hidden PDE models. The basic idea of the proposed PDE-Net is to learn differential operators by learning convolution kernels (filters), and apply neural networks or other machine learning methods to approximate the unknown nonlinear responses. Comparing with existing approaches, which either assume the form of the nonlinear response is known or fix certain finite difference approximations of differential operators, our approach has the most flexibility by learning both differential operators and the nonlinear responses. A special feature of the proposed PDE-Net is that all filters are properly constrained, which enables us to easily identify the governing PDE models while still maintaining the expressive and predictive power of the network. These constrains are carefully designed by fully exploiting the relation between the orders of differential operators and the orders of sum rules of filters (an important concept originated from wavelet theory). We also discuss relations of the PDE-Net with some existing networks in computer vision such as Network-In-Network (NIN) and Residual Neural Network (ResNet). Numerical experiments show that the PDE-Net has the potential to uncover the hidden PDE of the observed dynamics, and predict the dynamical behavior for a relatively long time, even in a noisy environment.

Figures (16)

• Figure 1: The schematic diagram of a $\delta t$-block.
• Figure 2: The schematic diagram of the PDE-Net: multiple $\delta t$-blocks.
• Figure 3: Prediction errors of the PDE-Net (orange) and Frozen-PDE-Net (blue) with $5\times 5$ (first row) and $7\times 7$ (second row) filters. In each plot, the horizontal axis indicates the time of prediction in the interval $(0,60\times\delta t]=(0,0.6]$, and the vertical axis shows the normalized errors. The banded curves indicate the 25% & 75% percentile of the normalized errors among 560 test samples.
• Figure 4: Long-time prediction for the PDE-Net with $7\times 7$ filters. The horizontal axis ranges in $(0,5]$. Time step $\delta t=0.01$.
• Figure 5: Images of the true dynamics and the predicted dynamics. The first row shows the images of the true dynamics. The second row shows the images of the predicted dynamics using the PDE-Net having 3 $\delta t$-blocks with $5\times 5$ and $7\times 7$ filters. Time step $\delta t=0.01$.

Theorems & Definitions (1)

• Definition 2.1: Order of Sum Rules