Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A neural network kernel decomposition for learning multiple steady states in parameterized dynamical systems

Yimeng Zhang, Alexander Cloninger, Bo Li, Xiaochuan Tian

TL;DR

This work tackles identifying parameter values that yield steady-state solutions in parameterized dynamical systems, and locating those steady states while assessing their linear stability, using an equation-free, data-driven PSNN framework. The core idea is to learn a target function $\Phi(U,\Theta)$ that peaks at steady states and, separately, $\Phi^{\rm s}(U,\Theta)$ that encodes stability, via a coupled neural network with a parameter subnetwork and a solution subnetwork connected through an inner product. The authors prove a universal approximation theorem for PSNNs based on a Mercer-kernel spectral decomposition, derive convergence and error bounds, and devise practical training and post-processing algorithms (including $K$-means clustering) to locate all steady states and generate phase diagrams across parameter regions. Numerical results on the Gray–Scott model demonstrate accurate phase-boundary prediction, robust solution localization, and resilience to incomplete data, validating the theory and highlighting the method’s efficiency over neural-network-free baselines. The approach provides a general, equation-free pathway for solving parameterized nonlinear systems of equations and identifying stable steady states with quantified uncertainty across parameter spaces.

Abstract

We develop a data-driven machine learning approach to identifying parameters with steady-state solutions, locating such solutions, and determining their linear stability for systems of ordinary differential equations and dynamical systems with parameters. Our approach first constructs target functions for these tasks, then designs a parameter-solution neural network (PSNN) that couples a parameter neural network and a solution neural network to approximate the target functions. We further develop efficient algorithms to train the PSNN and locate steady-state solutions. An approximation theory for the target functions with PSNN is developed based on kernel decomposition. Numerical results are reported to show that our approach is robust in finding solutions, identifying phase boundaries, and classifying solution stability across parameter regions. These numerical results also validate our analysis. While this study focuses on steady states of parameterized dynamical systems, our approach is equation-free and is applicable generally to finding solutions for parameterized nonlinear systems of algebraic equations. Some potential improvements and future work are discussed.

A neural network kernel decomposition for learning multiple steady states in parameterized dynamical systems

TL;DR

This work tackles identifying parameter values that yield steady-state solutions in parameterized dynamical systems, and locating those steady states while assessing their linear stability, using an equation-free, data-driven PSNN framework. The core idea is to learn a target function that peaks at steady states and, separately, that encodes stability, via a coupled neural network with a parameter subnetwork and a solution subnetwork connected through an inner product. The authors prove a universal approximation theorem for PSNNs based on a Mercer-kernel spectral decomposition, derive convergence and error bounds, and devise practical training and post-processing algorithms (including -means clustering) to locate all steady states and generate phase diagrams across parameter regions. Numerical results on the Gray–Scott model demonstrate accurate phase-boundary prediction, robust solution localization, and resilience to incomplete data, validating the theory and highlighting the method’s efficiency over neural-network-free baselines. The approach provides a general, equation-free pathway for solving parameterized nonlinear systems of equations and identifying stable steady states with quantified uncertainty across parameter spaces.

Abstract

We develop a data-driven machine learning approach to identifying parameters with steady-state solutions, locating such solutions, and determining their linear stability for systems of ordinary differential equations and dynamical systems with parameters. Our approach first constructs target functions for these tasks, then designs a parameter-solution neural network (PSNN) that couples a parameter neural network and a solution neural network to approximate the target functions. We further develop efficient algorithms to train the PSNN and locate steady-state solutions. An approximation theory for the target functions with PSNN is developed based on kernel decomposition. Numerical results are reported to show that our approach is robust in finding solutions, identifying phase boundaries, and classifying solution stability across parameter regions. These numerical results also validate our analysis. While this study focuses on steady states of parameterized dynamical systems, our approach is equation-free and is applicable generally to finding solutions for parameterized nonlinear systems of algebraic equations. Some potential improvements and future work are discussed.
Paper Structure (15 sections, 7 theorems, 52 equations, 9 figures, 1 table, 4 algorithms)

This paper contains 15 sections, 7 theorems, 52 equations, 9 figures, 1 table, 4 algorithms.

Table of Contents

  1. Introduction
  2. Parameter-Solution Neural Networks and Numerical Algorithms
  3. Problem formulation and assumptions
  4. Target functions
  5. The architecture of the parameter-solution neural network
  6. Training the parameter-solution neural network
  7. Locating solutions
  8. Approximation theory for the parameter-solution neural network
  9. Numerical Results
  10. The Gray--Scott model
  11. Convergence test
  12. Locating solutions and phase boundaries
  13. Incomplete data
  14. Conclusion
  15. Mean-shift algorithm

Key Result

Theorem 1

\newlabelthm:main_approximation0 Let $\Omega\subset\mathbb{R}^m$ be an open and bounded set and $D\subset \mathbb{R}^n$ be a hyperrectangle. Assume $\Phi \in L^2(D\times \Omega)$ satisfies that $\Phi (\cdot, x) \in C^{0, \alpha} (\overline{D})$ for each $x \in \Omega$ and $\Phi(y, \cdot) \in C^{0, More specifically, the above inequality can be attained with a parameter network $\Phi_{{\rm PNN}}:

Figures (9)

  • Figure 1: Schematic description of the structure of the parameter-solution neural network $\Phi_{\rm PSNN}$.
  • Figure 1: Test errors against the depth of the two sub-networks $\Phi_{\rm PNN}$ and $\Phi_{\rm SNN}$ for PSNN for different output dimensions $N$. Here the $x$-axis is the value of $L_1, L_2$ (with $L_1=L_2$). The $y$-axis gives the test error in $log$ scale. And curves in different colors depict the test error corresponding to different values of $N$.
  • Figure 2: Test errors against the depth with varying width for the PSNN with a fixed dimension $N = 8$ of the output vectors from the two sub-networks. The $x$-axis represents the value of $L_1$ (or $L_2$), and the $y$-axis gives the test error in $log$ scale. (a) $\Phi_{\rm SNN}$ is tested for $L_2=1,2,3,4$ and $W_2=5, 10, 15, 20,$ and $\Phi_{\rm PNN}$ has the fixed structure: $L_1=4, W_1=30.$ (b) $\Phi_{\rm PNN}$ is tested for $L_2=1,2,3,4,5$ and $W_2=5, 10, 15, 20, 30,$ and $\Phi_{\rm SNN}$ has the fixed structure: $L_1=3, W_1=20.$
  • Figure 3: The phase diagram for solution prediction. The algorithms are set up to forecast the number of solutions within range $\{0,1, ,\dots, 5\}.$ The blue points represent the parameter pairs that the algorithm recognized as no-solution, and the brown points, green points and orange points respectively correspond to 1-solution, 2-solution, 3-and-more-solution. The red curve represents the phase boundary of $\Omega_0$ and $\Omega_1$, which is given by the formula $f-4(f+k)^2=0$.
  • Figure 4: The Phase diagram for solution prediction with stability information. The algorithm is set to forecast the stability of learned solutions. The blue points, green points, brown points respectively represent 2-unstable-solution, 1-stable-1-unstable-solution, 2-stable-solution, and the black dashed curve plotted on $f\sqrt{f^2-4f(f+k)^2}+f^2-2(f+k)^3=0$ gives the boundary of $\Omega_{1,1}$ and $\Omega_{1,2}$.
  • ...and 4 more figures

Theorems & Definitions (18)

  • Remark 2.1
  • Remark 2.2
  • Remark 2.3
  • Theorem 1
  • Definition 2
  • Lemma 3
  • Proof 1
  • Definition 4
  • Proposition 5
  • Proof 2
  • ...and 8 more