Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunities

TL;DR

The paper addresses optimization and generalization challenges in physics-informed neural networks (PINNs) arising from multi-term, nonconvex losses and spectral biases. It advocates Evo-PINN, a framework that combines gradient-free evolutionary algorithms with gradient-based training to explore global optima, discover physics-biased architectures, and transfer knowledge across tasks. Key contributions include the use of Pareto-front multi-objective optimization, neuroevolution, transfer optimization, and evolutionary meta-learning (including Baldwinian-PINN) to improve convergence, uncertainty quantification, and generalization for physics-constrained learning. The work argues that such hybrid, evolution-inspired strategies can unlock scalable, data-efficient, and interpretable scientific AI with broad impact across simulation and discovery.

Abstract

Deep learning models trained on finite data lack a complete understanding of the physical world. On the other hand, physics-informed neural networks (PINNs) are infused with such knowledge through the incorporation of mathematically expressible laws of nature into their training loss function. By complying with physical laws, PINNs provide advantages over purely data-driven models in limited-data regimes and present as a promising route towards Physical AI. This feature has propelled them to the forefront of scientific machine learning, a domain characterized by scarce and costly data. However, the vision of accurate physics-informed learning comes with significant challenges. This work examines PINNs in terms of model optimization and generalization, shedding light on the need for new algorithmic advances to overcome issues pertaining to the training speed, precision, and generalizability of today's PINN models. Of particular interest are gradient-free evolutionary algorithms (EAs) for optimizing the uniquely complex loss landscapes arising in PINN training. Methods synergizing gradient descent and EAs for discovering bespoke neural architectures and balancing multiple terms in physics-informed learning objectives are positioned as important avenues for future research. Another exciting track is to cast EAs as a meta-learner of generalizable PINN models. To substantiate these proposed avenues, we further highlight results from recent literature to showcase the early success of such approaches in addressing the aforementioned challenges in PINN optimization and generalization.

Figures (10)

• Figure 1: Google Scholar search of the literature shows the popularity of PINN over the years between 2019 and 2024, and PINN studies in diverse scientific areas. Data collected in November 2024.
• Figure 2: Boosting scientific ML with evolutionary optimization of physics-informed neural networks (Evo-PINN).
• Figure 3: PINN handles both scientific simulation and scientific discovery. Example applications are: 1) accurate simulation of transient vortex shedding behind a cylinder obtained by learning a PINN for a single problem solely using the governing physics (2D transient N-S equations); and 2) inference of the wave propagation speed, $c$, given sparse observations (locations demarcated by $\boldsymbol{\times}$), by learning an inference PINN model with a partially specified 1D transient wave equation.
• Figure 4: Illustrative example showing how a PINN model can make accurate prediction by learning with physics laws (1D convection-diffusion equation) and few labeled data, whereas a conventional DNN model fails to learn the correct output.
• Figure 5: The plots contrast the local loss landscapes of PINN and DNN, for the convection-diffusion problem presented in Subsection \ref{['sec:pinn-advantages']}. After 100,000 training iterations, we project their corresponding loss values onto the plane spanned by the first two principal Hessian directions as per previously reported literature krishnapriyan2021characterizing to get an intuition for the loss landscape.