From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learning

TL;DR

The paper surveys recent advances in Physics-Informed Machine Learning, focusing on PINNs and the Kolmogorov-Arnold Network-based PIKANs as scalable, robust alternatives for solving ODEs/PDEs with sparse data. It provides a taxonomy of algorithmic developments across representation models, governing equations, and optimization strategies, including domain decomposition, separable architectures, autoregressively enhanced training, and adaptive loss balancing. The review also covers uncertainty quantification, theoretical underpinnings (NTK and information bottleneck perspectives), and a wide array of applications spanning biomedicine, fluids and solids mechanics, geophysics, dynamical systems, heat transfer, physics, chemical engineering, and beyond, along with computational frameworks and software. The discussion highlights practical impact and outlines future directions toward faster optimization, hyperparameter automation, higher accuracy, and closer integration with traditional numerical methods like FEM. Collectively, PINNs and PIKANs are presented as versatile, data-efficient tools with strong potential to reshape computational science across disciplines.

Abstract

Physics-Informed Neural Networks (PINNs) have emerged as a key tool in Scientific Machine Learning since their introduction in 2017, enabling the efficient solution of ordinary and partial differential equations using sparse measurements. Over the past few years, significant advancements have been made in the training and optimization of PINNs, covering aspects such as network architectures, adaptive refinement, domain decomposition, and the use of adaptive weights and activation functions. A notable recent development is the Physics-Informed Kolmogorov-Arnold Networks (PIKANS), which leverage a representation model originally proposed by Kolmogorov in 1957, offering a promising alternative to traditional PINNs. In this review, we provide a comprehensive overview of the latest advancements in PINNs, focusing on improvements in network design, feature expansion, optimization techniques, uncertainty quantification, and theoretical insights. We also survey key applications across a range of fields, including biomedicine, fluid and solid mechanics, geophysics, dynamical systems, heat transfer, chemical engineering, and beyond. Finally, we review computational frameworks and software tools developed by both academia and industry to support PINN research and applications.

Figures (16)

• Figure 1: Overview of the paper: Taxonomy of algorithmic developments (Section \ref{['algorithmic_developments']}), applications (Section \ref{['applications']}), uncertainty quantification (Section \ref{['uncertainty_quantification']}), and theory of PINNs(Section \ref{['theoretical_advances']}) is illustrated here.
• Figure 2: PIML Training components. A representation model provides an approximation $u$ of the ODE/PDE solution $\hat{u}$. The mismatch (residuals) between the approximated solution and the known components of the true solution, such as physical laws and boundary conditions, is calculated. These governing equations define a set of requirements that generate an optimization problem. This problem is reformulated as a loss function and then sent to an optimizer that updates the model parameters.
• Figure 3: Representation Model Enhancements: Input transformations improve the model's ability to impose periodic boundary conditions dong2021method, enhance expression capabilities guan2023dimension, and mitigate spectral bias through feature expansions wang2021eigenvector. Residual connections boost performance and accuracy for high-order derivatives required to enforce the PDE wang2024piratenets. The model architecture typically consists of layers, where each can be a perceptron (PINNs) or a KAN liu2024kan (PIKAN shukla2024comprehensive). PINN performance is further enhanced by modifications like weight normalization salimans2016weightwang2022random or adaptive activation functions jagtap2022deep. Output transformations enforce solution constraints, such as Dirichlet boundary conditions sukumar2022exact or divergence-free constraints wang2021understanding.
• Figure 4: Governing Equations. (a) Derivatives are required to enforce the ODE/PDE, boundary conditions (BCs), and possibly data. The most common method to obtain derivatives is through automatic differentiation baydin2018automatic. However, other methods have been proposed, such as finite differences lim2022physics, estimating derivatives as outputs gladstone2022fo, or stochastic dimension gradient descent hu2024tackling. (b) The physical laws are imposed using various operators, including fractional pang2019fpinns, stochastic yang2020physics, and even those allowing multiple solutions huang2022hompinns. Model performance can be enhanced through non-dimensionalization wang2022and, reformulations jin2021nsfnets, or approximations toscano2024invivo. (c) Residuals can be enforced either in their strong form raissi2019deep or weak form kharazmi2021hp. In the weak form, several types of test functions have been explored.
• Figure 5: Optimization Process Enhancements. The PIML problem involves satisfying multiple constraints in a given domain. For complex constraints, the problem can be simplified using sequential training or stacked training howard2023stacked. When dealing with complex domains (e.g., large, irregular), domain decomposition can be employed shukla2021parallel. (b) The loss function measures the cumulative error in the domain and, once discretized, comprises global weights wang2021understanding, local weights anagnostopoulos2024residual, a sampling method wu2023comprehensive, and a transformation function. (c) The optimizer updates the parameters using a line search method urban2024unveiling. The choice of the symmetric matrix defines the optimization method, such as gradient descent, Adam, L-BFGS, or Newton’s method. Several approaches have improved base performance, ensuring conflict-free updates liu2024config, or by using alternative methods like non-dominated genetic algorithms akhter2024common, or particle swarm optimization davi2022pso.