Machine Learning with Physics Knowledge for Prediction: A Survey

TL;DR

This survey addresses how machine learning can be made reliable for physical prediction by injecting physics through architecture, losses, and data priors, with a focus on ODEs and PDEs. It surveys families of methods including PINNs, neural ODEs/DAEs, Hamiltonian/Lagrangian and port-Hamiltonian networks, neural operators (DeepONet, FNO), and data-driven priors via multi-task learning, meta-learning, and neural processes. Key contributions include a structured taxonomy, discussion of industrial relevance, and a comprehensive view of open-source ecosystems for physics-informed ML. The work highlights practical challenges (training stability, sampling, extrapolation), emphasizes uncertainty quantification, and envisions foundation-model-scale physics ML as a future direction for scalable, data-efficient predictive modeling across science and industry.

Abstract

This survey examines the broad suite of methods and models for combining machine learning with physics knowledge for prediction and forecast, with a focus on partial differential equations. These methods have attracted significant interest due to their potential impact on advancing scientific research and industrial practices by improving predictive models with small- or large-scale datasets and expressive predictive models with useful inductive biases. The survey has two parts. The first considers incorporating physics knowledge on an architectural level through objective functions, structured predictive models, and data augmentation. The second considers data as physics knowledge, which motivates looking at multi-task, meta, and contextual learning as an alternative approach to incorporating physics knowledge in a data-driven fashion. Finally, we also provide an industrial perspective on the application of these methods and a survey of the open-source ecosystem for physics-informed machine learning.

Figures (8)

• Figure 1: This survey covers the diverse ways physics knowledge can be incorporated into ML across datasets ${\mathcal{D}}$, objectives ${\mathcal{L}}$ and models ${\bm{u}}_{\bm{\theta}}$. Physics-informed models are architectures and prediction techniques informed by domain knowledge, covered in sections \ref{['sec:node']}, \ref{['sec:equations']}, and \ref{['sec:lvm']}. Learning arbitrary models with physics-informed losses is discussed in sections \ref{['sec:pinns']} and \ref{['sec:sysid']}. Data augmentation can also incorporate physics knowledge to encode invariances (section \ref{['sec:aug']}). Data-driven learning across multiple datasets and experiments includes neural operators (section \ref{['sec:no']}), multi-task learning (section \ref{['sec:multitask']}) and meta-learning. Finally, contextual data is another data-driven approach to incorporate additional domain knowledge, as seen in operator learning (section \ref{['sec:no']}) and neural processes (section \ref{['sec:np']}).
• Figure 2: A schematic of deep Lagrangian networks for rigid body physics. Through careful use of automatic differentiation, the Lagrangian loss can be constructed from the state variables and rigid body terms, avoiding the need to differentiate the model with respect to time. The predicted inertial matrix ${\bm{M}}({\bm{q}})$ is also guaranteed to be positive semi-definite through careful parameterization.
• Figure 3: A schematic of physics-informed neural networks, detailing the relationship between the approximated model of the solution ${\bm{u}}_{\bm{\theta}}({\bm{x}})$ and the physics-informed loss terms, where the integral over the domain has been replaced with a Monte Carlo approximation. Using forward-mode automatic differentiation, the required gradients of the solution are used to minimize the residual PDE error term (blue), while the boundary condition here only uses the solution values (magenta). However, model gradients may be required for the boundary loss, depending on how the boundary condition is defined.
• Figure 4: (Left) A schematic of operator networks that learn a functional mapping from the inputs $a$ to the PDE solution ${\bm{u}}$ at a designated point ${\bm{x}}$. (Middle) DeepONets can approximate any nonlinear operator by learning a finite-dimensional mapping $\hat{{\bm{u}}} = {\bm{\varphi}}_{\bm{\theta}}(\hat{{\bm{a}}})$ in a latent space which is spanned by linear encodings $\hat{{\bm{a}}} = {\bm{e}}_{\bm{\theta}}({\bm{a}})$ and linear decoding ${\bm{u}} = {\bm{g}}_{\bm{\theta}}(\hat{{\bm{u}}}, {\bm{x}})$. (Right) Neural operators directly map a discretized representation of the input ${\bm{a}}_{1:N}$ to the solution at an arbitrary discretization ${\bm{u}}_{1:M}$.
• Figure 5: Schematic overview of the Koopman operator theory. Left: Other than analyzing the evolution of a dynamical system on ${\mathcal{X}}$, the theory studies a linear but potentially infinite-dimensional operator on ${\mathcal{F}}\in\{{\mathbb{C}},{\mathbb{R}}\}$. Right: The graphical model provides an overview of how recent methods often address Koopman theory. Given the observations $\{{\bm{u}}_i\}^T_{i=0}$, the objective is twofold: (i) finding a feature representation transforming the observation into a latent coordinate system, and (ii) approximating the Koopman operator based on latent variables $\{{\bm{z}}_i\}^T_{i=0}$ assuming linear latent dynamics.