Discovering Physics-Informed Neural Networks Model for Solving Partial Differential Equations through Evolutionary Computation

TL;DR

The study tackles the manual design bottleneck in physics-informed neural networks (PINNs) for solving PDEs by introducing an evolutionary computation framework that simultaneously searches network structure and novel parametric activation functions, encoded as binary trees. A dynamic DPSTE strategy expands exploration early and concentrates evaluation on promising models to improve convergence speed and accuracy. Across Klein-Gordon, Burgers, and Lamé equations, evo-w/-DPSTE consistently yields superior approximation quality and faster convergence, with strong generalization to changes in source terms, initial/boundary conditions, coefficients, and domain. The approach expands the PINN design space and provides a practical, automated tool for robust PDE solving, with public code support.

Abstract

In recent years, the researches about solving partial differential equations (PDEs) based on artificial neural network have attracted considerable attention. In these researches, the neural network models are usually designed depend on human experience or trial and error. Despite the emergence of several model searching methods, these methods primarily concentrate on optimizing the hyperparameters of fully connected neural network model based on the framework of physics-informed neural networks (PINNs), and the corresponding search spaces are relatively restricted, thereby limiting the exploration of superior models. This article proposes an evolutionary computation method aimed at discovering the PINNs model with higher approximation accuracy and faster convergence rate. In addition to searching the numbers of layers and neurons per hidden layer, this method concurrently explores the optimal shortcut connections between the layers and the novel parametric activation functions expressed by the binary trees. In evolution, the strategy about dynamic population size and training epochs (DPSTE) is adopted, which significantly increases the number of models to be explored and facilitates the discovery of models with fast convergence rate. In experiments, the performance of different models that are searched through Bayesian optimization, random search and evolution is compared in solving Klein-Gordon, Burgers, and Lamé equations. The experimental results affirm that the models discovered by the proposed evolutionary computation method generally exhibit superior approximation accuracy and convergence rate, and these models also show commendable generalization performance with respect to the source term, initial and boundary conditions, equation coefficient and computational domain. The corresponding code is available at https://github.com/MathBon/Discover-PINNs-Model.

Figures (17)

• Figure 1: Approximation accuracy and convergence history of relative $L_2$ error about different models in solving (a) Klein-Gordon, (b) Burgers, and (c) Lamé equations. Each curve of convergence history is the average result of ten independent trials. The error in solving Lamé equations is about $u$. The five ways of shortcut connections are labeled as: I:none, II:[0-2,2-4,4-6,6-8], III:[0-1,1-3,4-6,7-8], IV:[1-4,4-6,6-8], V:[0-3,4-7], where '0-2' indicates the connection between the inputs of first layer and third layer, and so forth. 'tanh*' and 'swish*' denote layer-wise parametric activation functions tanh($\alpha \cdot x$) and $x$$\cdot$ sigmoid($\alpha \cdot x$).
• Figure 2: The examples about the general forms of structure and activation function. The structure in (a) has 9 layers with shortcut connections: [0-2,2-4,5-8]. The unary and binary operators adopted in the activation function are listed in Table \ref{['tab:operators']}.
• Figure 3: Genetic encoding of the neural network model and the corresponding crossover and mutation operations.
• Figure 4: Mutation about the structure of neural network model.
• Figure 5: Mutation about activation function of neural network model.