HomPINNs: homotopy physics-informed neural networks for solving the inverse problems of nonlinear differential equations with multiple solutions

TL;DR

The paper tackles inverse problems for nonlinear differential equations that admit multiple solutions and unlabeled observations. It introduces HomPINNs, a framework that fuses homotopy continuation with physics-informed neural networks to identify unknown parameters and recover multiple solution branches by gradually shifting weight from enforcing the DE to fitting the data, via the exponentially decaying homotopy parameter $\alpha_k=\alpha_0 r^{k-1}$. The method uses an M-output NN where each $\hat{u}_m$ represents a candidate solution and employs a data-fit term $\sum_i \min_m \|\hat{u}_m(\boldsymbol x_i)-u_i\|^2$ alongside a DE residual term to handle unlabeled observations and enforce physical constraints. Demonstrations on 1D problems with 2 and 7 solutions and a 2D Gray–Scott system show that HomPINNs can identify DE parameters and recover all solutions, even when only partial or unlabeled data are available, highlighting its robustness and potential for complex scientific applications. The work suggests broad impact for modeling and solving inverse problems in physics, chemistry, and biology where non-uniqueness and bifurcations are common.

Abstract

Due to the complex behavior arising from non-uniqueness, symmetry, and bifurcations in the solution space, solving inverse problems of nonlinear differential equations (DEs) with multiple solutions is a challenging task. To address this, we propose homotopy physics-informed neural networks (HomPINNs), a novel framework that leverages homotopy continuation and neural networks (NNs) to solve inverse problems. The proposed framework begins with the use of NNs to simultaneously approximate unlabeled observations across diverse solutions while adhering to DE constraints. Through homotopy continuation, the proposed method solves the inverse problem by tracing the observations and identifying multiple solutions. The experiments involve testing the performance of the proposed method on one-dimensional DEs and applying it to solve a two-dimensional Gray-Scott simulation. Our findings demonstrate that the proposed method is scalable and adaptable, providing an effective solution for solving DEs with multiple solutions and unknown parameters. Moreover, it has significant potential for various applications in scientific computing, such as modeling complex systems and solving inverse problems in physics, chemistry, biology, etc.

Figures (16)

• Figure 1: The neural network structure of HomPINN for the inverse problems: given the observations $\left\{{\boldsymbol x}_i, u_i\right\}_{i=1}^{N_o}$ and collocation points $\{{\boldsymbol x}_j\}^{N_c}_{j=1}$, HomPINN makes prediction $\boldsymbol{\hat{u}}$, where $\boldsymbol{\hat{u}}$ is a concatenation of $\hat{u}_m$ ($m=1, 2, \cdots, M$). Here $M$ is an estimated total number of solutions of (\ref{['equation_pde1']}), and $\hat{u}_m$ (or $\hat{u}_m({\boldsymbol {x}_i}; \boldsymbol \theta,\boldsymbol \lambda)$) approximates one of the solutions of (\ref{['equation_pde1']}). Two constraints (purple rectangular boxes) are considered here to constrain the approximations of HomPINNs during the optimization.
• Figure 2: Schematic of HomPINNs: a NN at step $k$ ($k=1,2,\cdots, K$) is constructed as Fig. \ref{['fig_msnn']}, where the network parameters are initialized by a previous well-trained NN at step $k-1$. Then the observations and collocation points are used to train the NN at each homotopy step. The homotopy process starts from step $1$ and ends in step $K$. The unknown parameter $\boldsymbol \lambda^*$, the optimal parameters $\boldsymbol \theta^*$, and the solutions of (\ref{['equation_pde1']}) will be obtained finally.
• Figure 3: Observations sampled from the two solutions of (\ref{['equation_ex1']}) and approximates made from HomPINNs. Here the x-axis is $x$, and the y-axis is $u$. The black circles are the sampled observations. The red solid line with squares and the blue solid line with stars are the approximated $\hat{u}_{m}(\cdot)$ from HomPINNs. The correct parameter value $\lambda=1.20$, and with the correct estimate of $M=2$, the identified DE parameter given by HomPINNs $\lambda$ is $1.200$.
• Figure 4: Training loss during the homotopy steps with the selection of $M=1,2,3,4$. Here the x-axis indicates the number of homotopy processes performed by HomPINNs, where $\alpha_k$ ($k=1, 2, \cdots, 11$) is the index of each homotopy step. The y-axis is the training loss in the logarithmic scales.
• Figure 5: The trajectory of each homotopy training session over its associated number of homotopy steps. The x-axis corresponds to the homotopy step index, while the y-axis represents the values of the unknown DE parameter $\lambda$. Left: Individual trajectories are depicted as gray dashed lines, each of which illustrates the temporal evolution of $\lambda$ during the homotopy training process. Right: For the violin chart, the width of the PDF serves as an indicator of relative frequency for each observed $\lambda$ value. The top and bottom boundaries of the inner box correspond to the third and first quantiles, respectively, and a white circle within the box signifies the median value. Thin black lines extending from the box indicate the lambda's minimum and maximum values. The ground truth: $\lambda=1.20$.

Theorems & Definitions (2)

• Remark 1
• Remark 2