An Unsupervised Network Architecture Search Method for Solving Partial Differential Equations

TL;DR

The paper tackles the challenge of selecting optimal neural network architectures for physics-informed neural networks (PINNs) by introducing PINN-DARTS, an unsupervised neural-architecture-search method based on differentiable architecture search (DARTS). It formulates the search over varying layer depths and widths within a PINN, optimizing architecture and weights concurrently in a PDE-residual-guided framework, and incorporates DARTS+, FairDARTS variants to mitigate skip-connection bias. Across Poisson, Heat, Wave, and Burgers equations, PINN-DARTS and its variants achieve higher architectural accuracy and competitive to superior solution accuracy with notably shorter search times, with PINN-DARTS+ often delivering the best trade-off, especially for complex or nonlinear PDEs. The results suggest problem-tailored, uneven-width architectures can outperform uniform-width designs, highlighting the practical impact of integrated NAS within PINN for efficient and accurate PDE solving in scientific computing.

Abstract

Solving partial differential equations (PDEs) has been indispensable in scientific and engineering applications. Recently, deep learning methods have been widely used to solve high-dimensional problems, one of which is the physics-informed neural network (PINN). Typically, a deep learning method has three main components: a neural network, a loss function, and an optimizer. While the construction of the loss function is rooted in the definition of solution space, how to choose a optimal neural network is somewhat ad hoc, leaving much room for improvement. In the framework of PINN, we propose an unsupervised network architecture search method for solving PDEs, termed PINN-DARTS, which applies the differentiable architecture search (DARTS) to find the optimal network architecture structure in a given set of neural networks. In this set, the number of layers and the number of neurons in each layer can change. In the searching phase, both network and architecture parameters are updated simultaneously, so the running time is close to that of PINN with a pre-determined network structure. Unlike available works, our approach is unsupervised and purely based on the PDE residual without any prior usage of solutions. PINN-DARTS outputs the optimal network structure as well as the associated numerical solution. The performance of PINN-DARTS is verified on several benchmark PDEs, including elliptic, parabolic, wave, and Burgers' equations. Compared to traditional architecture search methods, PINN-DARTS achieves significantly higher architectural accuracy. Another interesting observation is that both the solution complexity and the PDE type have a prominent impact on the optimal network architecture. Our study suggests that architectures with uneven widths from layer to layer may have superior performance across different solution complexities and different PDE types.

Figures (11)

• Figure 1: The framework of FNN. $x_1$, $x_2$ are the input of FNN and $y$ is the network output. $h^{(i)}$ represents hidden layer $i$.
• Figure 2: A cell of DARTS. All edges point from preceding nodes to subsequent nodes. There are multiple edges between two nodes, each corresponding to a candidate operation.
• Figure 3: The structure of PINN-DARTS. Each node only has edges with its adjacent nodes. The candidate operations consist of linear layers with different widths and the skip-connection operation. The candidate operations are weighted by $\alpha_{\mathcal{O}}^{(i-1,i)}$. $h_i$ is the output tensor of the hidden layer corresponding to node $i$, and it is the sum of weighted candidate operation inputs.
• Figure 4: Spearman's rank correlation coefficients of PINN loss and relative $L^2$ error for a series of architectures with different widths and depths for the Poisson equation with simple and complex solutions.
• Figure 5: Comparison of PINN-DARTS methods and traditional methods for the Poisson equation with simple and complex solutions.