Evolutionary Optimization of Physics-Informed Neural Networks: Advancing Generalizability by the Baldwin Effect

TL;DR

PINNs promise physics‑aware predictions but struggle to generalize across unseen physics tasks. Baldwinian-PINN combines Baldwinian neuroevolution with a two-stage stochastic program to pre-wire learning biases into hidden layers and perform rapid, physics‑informed lifetime learning in the final layer via closed-form least-squares. Across linear and nonlinear ODE/PDE families, Baldwinian-PINNs achieve orders‑of‑magnitude speedups and substantial accuracy gains over gradient-based meta-learning, enabling fast, physics-consistent predictions for diverse scenarios. This framework offers a scalable path to generalizable, continual physics solvers with potential applications in design, what-if analyses, and beyond.

Abstract

Physics-informed neural networks (PINNs) are at the forefront of scientific machine learning, making possible the creation of machine intelligence that is cognizant of physical laws and able to accurately simulate them. However, today's PINNs are often trained for a single physics task and require computationally expensive re-training for each new task, even for tasks from similar physics domains. To address this limitation, this paper proposes a pioneering approach to advance the generalizability of PINNs through the framework of Baldwinian evolution. Drawing inspiration from the neurodevelopment of precocial species that have evolved to learn, predict and react quickly to their environment, we envision PINNs that are pre-wired with connection strengths inducing strong biases towards efficient learning of physics. A novel two-stage stochastic programming formulation coupling evolutionary selection pressure (based on proficiency over a distribution of physics tasks) with lifetime learning (to specialize on a sampled subset of those tasks) is proposed to instantiate the Baldwin effect. The evolved Baldwinian-PINNs demonstrate fast and physics-compliant prediction capabilities across a range of empirically challenging problem instances with more than an order of magnitude improvement in prediction accuracy at a fraction of the computation cost compared to state-of-the-art gradient-based meta-learning methods. For example, when solving the diffusion-reaction equation, a 70x improvement in accuracy was obtained while taking 700x less computational time. This paper thus marks a leap forward in the meta-learning of PINNs as generalizable physics solvers. Sample codes are available at https://github.com/chiuph/Baldwinian-PINN.

Figures (7)

• Figure 1: Schematic diagram of (a) Baldwinian evolution in nature and (b) evolving machine intelligence for learning physics with Baldwinian-PINNs. In nature, the Baldwin effect describes how learned traits are eventually reinforced in the genetic makeup of a population of organisms through natural selection. Equivalently, a population of Baldwinian-PINNs evolves over generations by being exposed to a broad distribution of physics tasks, gradually reinforcing traits promoting accurate physics learning into their genetic makeup. The evolved Baldwinian-PINNs are inherently equipped with strong learning biases to accurately solve any physics tasks over a broad task distribution.
• Figure 2: (a) Meta-learning PINN with Baldwinian neuroevolution (right) versus MAML (left). In MAML, the initial weights $\vb*{W}$ are learned using gradient-based method, such that they can be quickly fine-tuned (physics-informed learning) on new tasks. Although the task-specific fine-tuning is limited to one or a few gradient descent updates during training, such amount of fine-tuning is usually insufficient for a PINN at test time. In Baldwinian neuroevolution, the weight distribution in the pre-final nonlinear hidden layers $\vb*{\tilde{w}}$ and learning hyperparameters $\lambda$ are jointly evolved. The task-specific physics-informed learning is performed on the final layer weights $w$ (segregated from $\vb*{\tilde{w}}$) with a 1-step Tikhonov regularization operation at both training and test time (for linear ODE/PDEs). (b) Schematic of Baldwinian-PINNs architecture used in present study and procedure to obtain nonlinear hidden layers’ weights $\vb*{\tilde{w}}$’s from the evolved network hyperparameter $\theta$ for learning task-specific outputs.
• Figure 3: (a) Solution of 20 individual Baldwinian-PINN models sampled from the CMA-ES search distribution (initial std. = 1), for unseen convection-diffusion tasks $\alpha = \{4,27,108\}$. The task is more challenging with increasing $\alpha$. Baldwinian neuroevolution is effective for evolving good Baldwinian-PINN models which can generalize across different difficulties. (b) Baldwinian neuroevolution demonstrates effective LSE and MSE convergences on convection-diffusion problem, for different CMA-ES initial std. values (best std. = 1). The bold lines indicate their median convergence path from 5 individual runs, and the shaded areas indicate their interquartile ranges.
• Figure 4: (a) Schematic to illustrate new tasks arising from family of linear PDEs problem: S1 change to new PDE and IC profile for $t\in[0,2]$ (same time domain as train tasks), and S2 projection to longer time domain $t\in[0,4]$. (b) Solution for unseen linear PDE tasks obtained by best evolved Baldwinian-PINN sampled from the center of CMA-ES search distribution after 500 iterations with initial std. = 5 (they are visually indistinguishable from the ground truth). (c) The mean MSE over $n=87$ test tasks for 2 test scenarios described in (a) are below $5\mathrm{e}{-5}$. (d) Baldwinian neuroevolution demonstrates effective LSE and MSE convergences for different CMA-ES initial std. values, with a superior performance given by std. = 5 and 10. The bold lines indicate their median convergence path from 5 individual runs, and the shaded areas indicate their interquartile ranges.
• Figure 5: (a) Solution of 20 individual Baldwinian-PINN models sampled from the CMA-ES search distribution (initial std. = 0.5), for unseen kinematics tasks. (c) Solution of Baldwinian-PINN model sampled from the center of NES search distribution after 400 iterations (initial std. = 0.5), for unseen fluid flow (N-S equations) tasks. The velocity magnitude predictions are visually indistinguishable from the ground truth, even as Reynolds number (Re) increases. Their mid‑section $u$ and $v$-velocity profiles (dashed lines) show excellent agreement with the classic benchmark results (marked points) from Ghia ghia1982high. Baldwinian neuroevolution demonstrates effective LSE and MSE convergence on both (b) kinematics and (d) fluid flow problem, for different CMA-ES initial std. values. The bold lines indicate their median convergence path from 5 individual runs, and the shaded areas indicate their interquartile ranges.