LordNet: An Efficient Neural Network for Learning to Solve Parametric Partial Differential Equations without Simulated Data

TL;DR

The paper addresses the data-efficiency challenge in learning solution operators for parametric PDEs by leveraging a physics-constrained mean squared residual (MSR) loss, and identifies long-range entanglements in the MSR signal as the key learning signal. It introduces LordNet, a low-rank, multi-channel architecture that mimics solver-like matrix operations to capture diverse long-range dependencies with substantially fewer parameters and lower computation. Through experiments on Poisson's equation and 2D/3D Navier–Stokes problems, LordNet consistently outperforms conventional neural PDE solvers and achieves up to 40× speedups over traditional GPU solvers while maintaining high accuracy. The results demonstrate that MSR-driven training combined with LordNet offers a practical, data-efficient path to accurate, scalable neural operators for parametric PDEs, including complex 3D flows.

Abstract

Neural operators, as a powerful approximation to the non-linear operators between infinite-dimensional function spaces, have proved to be promising in accelerating the solution of partial differential equations (PDE). However, it requires a large amount of simulated data, which can be costly to collect. This can be avoided by learning physics from the physics-constrained loss, which we refer to it as mean squared residual (MSR) loss constructed by the discretized PDE. We investigate the physical information in the MSR loss, which we called long-range entanglements, and identify the challenge that the neural network requires the capacity to model the long-range entanglements in the spatial domain of the PDE, whose patterns vary in different PDEs. To tackle the challenge, we propose LordNet, a tunable and efficient neural network for modeling various entanglements. Inspired by the traditional solvers, LordNet models the long-range entanglements with a series of matrix multiplications, which can be seen as the low-rank approximation to the general fully-connected layers and extracts the dominant pattern with reduced computational cost. The experiments on solving Poisson's equation and (2D and 3D) Navier-Stokes equation demonstrate that the long-range entanglements from the MSR loss can be well modeled by the LordNet, yielding better accuracy and generalization ability than other neural networks. The results show that the Lordnet can be $40\times$ faster than traditional PDE solvers. In addition, LordNet outperforms other modern neural network architectures in accuracy and efficiency with the smallest parameter size.

Figures (7)

• Figure 1: The diagram of Long-range entanglements: taking Poisson's equation as an example. The vectors $\hat{a}$ and $\hat{L}_i$ are reshaped into the matrix form to highlight the contribution of each point in the spatial domain to the blue dots.
• Figure 2: An illustration of LordNet architecture.
• Figure 3: We present solutions to the Navier-Stokes equations in a lid-driven cavity context, employing MSR loss across various neural network architectures. The top row displays the numerical simulation results, while the subsequent rows from top to bottom illustrate the predictions generated by LordNet, FNO, Swin Transformer, and ResNet, respectively.
• Figure 4: Randomly selected examples of training domains consisting of 128$\times$64$\times$64 voxels. The locations of the obstacles are also randomly chosen, and the inflow and outflow boundaries are on the left and right sides of the domains.
• Figure 5: Streamlines and pressure fields of flow around a square rod at different Reynolds numbers. The diameter of the rod is 16. a) is the result of $\rho=0.5, \mu=3.0$, resulting in a Reynolds number of 48, while b) is the result of $rho=0.1, \mu=8.0$, resulting in a Reynolds number of 640. The results are generated by LordNet.