Learning Data-Efficient and Generalizable Neural Operators via Fundamental Physics Knowledge

TL;DR

This work tackles the data inefficiency and limited generalization of neural operators for PDEs by introducing a multiphysics training framework that jointly learns from the original PDEs and decomposed basic forms representing fundamental physics. By identifying and enforcing basic physical terms such as diffusion, advection, and linearized KS dynamics, the authors create a principled auxiliary task that preserves essential dynamics while enabling cheaper simulations. The approach yields consistent gains in data efficiency, long-term predictive stability, and out-of-distribution as well as synthetic-to-real generalization across 1D/2D/3D PDEs and multiple neural-operator architectures, including FNO and Transformer-based models. These findings highlight the value of explicit physics priors as inductive biases in SciML, offering a scalable route to more reliable surrogate models for complex physical systems, with accompanying code and models released publicly.

Abstract

Recent advances in scientific machine learning (SciML) have enabled neural operators (NOs) to serve as powerful surrogates for modeling the dynamic evolution of physical systems governed by partial differential equations (PDEs). While existing approaches focus primarily on learning simulations from the target PDE, they often overlook more fundamental physical principles underlying these equations. Inspired by how numerical solvers are compatible with simulations of different settings of PDEs, we propose a multiphysics training framework that jointly learns from both the original PDEs and their simplified basic forms. Our framework enhances data efficiency, reduces predictive errors, and improves out-of-distribution (OOD) generalization, particularly in scenarios involving shifts of physical parameters and synthetic-to-real transfer. Our method is architecture-agnostic and demonstrates consistent improvements in normalized root mean square error (nRMSE) across a wide range of 1D/2D/3D PDE problems. Through extensive experiments, we show that explicit incorporation of fundamental physics knowledge significantly strengthens the generalization ability of neural operators. We will release models and codes at https://sites.google.com/view/sciml-fundemental-pde.

Figures (18)

• Figure 1: Can SciML models (e.g., neural operators trained on advanced PDEs) also understand fundamental physics knowledge (basic terms like diffusion, advection)?
• Figure 2: On 2D incompressible Navier-Stokes, neural operators and SciML foundation models (MPP mccabe2023multiple, DPOT hao2024dpot, Hyena patil2023hyenaneuraloperatorpartial) exhibit correlated yet worse performance on fundamental physics (x-axis).
• Figure 3: Overview of our method. Decomposed PDEs encode rich fundamental physical knowledge and introduce cheaper simulations. By jointly training on both the full PDE and its decomposed basic form, we bring multiple benefits to neural operators.
• Figure 4: Visualizations of simulations of PDEs and their decomposed basic forms (Section \ref{['sec:pdes']}). From left to right: Diffusion-Reaction (activator concentration), 2D Navier-Stokes (fluid velocity), 3D Navier-Stokes (smoke density), and Kuramoto-Sivashinsky (perturbation amplitude). Basic PDE forms are used for training neural operators with fundamental physics knowledge (Section \ref{['sec:joint_learning']}), and the OOD settings are used for evaluating the generalization of neural operators (Section \ref{['sec:exp_ood']}). $D_v, D_u$: diffusion coefficients (Equation \ref{['eq:diffusion_reaction']}). $\nu$: viscosity (Equation \ref{['eq:ns']}). $a,b,k_1,k_2$: magnitudes and wavenumbers (Equation \ref{['eq:ks_init']}).
• Figure 5: Joint training neural operators on data of the original PDE and the basic form improves performance and data efficiency. Y-axis: normalized RMSE. X-axis: simulation costs (seconds).