Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Chaos-Free Networks are Stable Recurrent Neural Networks

Stefano De Carli, Davide Previtali, Mirko Mazzoleni, Fabio Previdi

Abstract

Gated Recurrent Neural Networks (RNNs) are widely used for nonlinear system identification due to their high accuracy, although they often exhibit complex, chaotic dynamics that are difficult to analyze. This paper investigates the system-theoretic properties of the Chaos-Free Network (CFN), an architecture originally proposed to eliminate the chaotic behavior found in standard gated RNNs. First, we formally prove that the CFN satisfies Input-to-State Stability (ISS) by design. However, we demonstrate that ensuring Incremental ISS (delta-ISS) still requires specific parametric constraints on the CFN architecture. Then, to address this, we introduce the Decoupled-Gate Network (DGN), a novel structural variant of the CFN that removes internal state connections in the gating mechanisms. Finally, we prove that the DGN unconditionally satisfies the delta-ISS property, providing an incrementally stable architecture for identifying nonlinear dynamical systems without requiring complex network training modifications. Numerical results confirm that the DGN maintains the modeling capabilities of standard architectures while adhering to these rigorous stability guarantees.

Chaos-Free Networks are Stable Recurrent Neural Networks

Abstract

Gated Recurrent Neural Networks (RNNs) are widely used for nonlinear system identification due to their high accuracy, although they often exhibit complex, chaotic dynamics that are difficult to analyze. This paper investigates the system-theoretic properties of the Chaos-Free Network (CFN), an architecture originally proposed to eliminate the chaotic behavior found in standard gated RNNs. First, we formally prove that the CFN satisfies Input-to-State Stability (ISS) by design. However, we demonstrate that ensuring Incremental ISS (delta-ISS) still requires specific parametric constraints on the CFN architecture. Then, to address this, we introduce the Decoupled-Gate Network (DGN), a novel structural variant of the CFN that removes internal state connections in the gating mechanisms. Finally, we prove that the DGN unconditionally satisfies the delta-ISS property, providing an incrementally stable architecture for identifying nonlinear dynamical systems without requiring complex network training modifications. Numerical results confirm that the DGN maintains the modeling capabilities of standard architectures while adhering to these rigorous stability guarantees.
Paper Structure (16 sections, 6 theorems, 28 equations, 2 figures)

This paper contains 16 sections, 6 theorems, 28 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Preliminaries and problem statement
  3. Notation and preliminaries
  4. Problem statement
  5. Chaos-Free and Decoupled-Gate Networks
  6. The Chaos-Free Network (CFN)
  7. The proposed Decoupled-Gate Network (DGN)
  8. Stability properties
  9. Stability of CFN architectures
  10. Extension to DGN architectures
  11. Numerical results
  12. Training configuration
  13. pH Reactor
  14. Four-Tank System
  15. Conclusion
  16. ...and 1 more sections

Key Result

Proposition 1

For any $l \in \mathcal{L}$, the set $\mathcal{H}_{\mathrm{inv}}^{(l)} := \left[-2, 2\right]^{n_{h}^{(l)}}$ is a forward invariant compact set for the layer dynamics in eq:CFN_layer_state_update. Thus, for any initial condition $\boldsymbol{h}_{0}^{(l)} \in \mathcal{H}_{\mathrm{inv}}^{(l)}$, the hid

Figures (2)

  • Figure 1: Single-layer dynamics for the CFN and the DGN at time $k$. The dashed paths represent the recurrent connections in the gates. The CFN includes these connections, whereas the DGN removes them to guarantee unconditional stability.
  • Figure 2: Model predictions on the test dataset $\mathcal{D}_{\mathrm{tst}}$ for the pH Reactor (top) and the Four-Tank System (bottom). The plots compare the ground truth (dashed black line) against the predictions of the CFN (thick orange line) and the proposed DGN (thin blue line).

Theorems & Definitions (9)

  • Definition 1: ISS jiang_input--state_2001terzi_learning_2021
  • Definition 2: $\delta$ISS bayerDiscretetimeIncrementalISS2013terzi_learning_2021
  • Remark 1
  • Proposition 1: Forward invariant set of $\boldsymbol{h}_k^{(l)}$
  • Theorem 1: ISS of a CFN layer
  • Theorem 2: $\delta$ISS of a CFN layer
  • Theorem 3: ISS of a CFN
  • Theorem 4: $\delta$ISS of a CFN
  • Theorem 5: $\delta$ISS of a DGN