Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Explicit construction of recurrent neural networks effectively approximating discrete dynamical systems

Chikara Nakayama, Tsuyoshi Yoneda

TL;DR

The paper addresses the explicit construction of recurrent neural networks that effectively approximate discrete dynamical systems in delay coordinates. It adopts a discretization strategy of the state range and a dictionary based on key value pairs to encode L-step history patterns, then algebraically constructs RNN weights via a regular matrix X to realize the target map. A main result guarantees that for any large grid size K and constant C > 1 there exists N ≤ K^L and explicit weights W, W_in, W_out such that the RNN output satisfies a provable bound |ŷ(t) - y(t)| ≤ (2t+1) exp(λ t) sqrt(L) C / K for t ≥ 0, linking the approximation error to the maximum Lyapunov exponent λ. The construction uses adjoint permutation operators and a carefully chosen h to ensure X is regular, enabling an exact reproduction of an auxiliary sequence y^* and thus effective re-expression of the original dynamical system. This work provides an explicit, algebraic bridge between discrete dynamical systems and RNNs with controlled stability guarantees, contributing to the understanding of how Lyapunov dynamics can be captured by recurrent architectures.

Abstract

We consider arbitrary bounded discrete time series originating from dynamical system with recursivity. More precisely, we provide an explicit construction of recurrent neural networks which effectively approximate the corresponding discrete dynamical systems.

Explicit construction of recurrent neural networks effectively approximating discrete dynamical systems

TL;DR

The paper addresses the explicit construction of recurrent neural networks that effectively approximate discrete dynamical systems in delay coordinates. It adopts a discretization strategy of the state range and a dictionary based on key value pairs to encode L-step history patterns, then algebraically constructs RNN weights via a regular matrix X to realize the target map. A main result guarantees that for any large grid size K and constant C > 1 there exists N ≤ K^L and explicit weights W, W_in, W_out such that the RNN output satisfies a provable bound |ŷ(t) - y(t)| ≤ (2t+1) exp(λ t) sqrt(L) C / K for t ≥ 0, linking the approximation error to the maximum Lyapunov exponent λ. The construction uses adjoint permutation operators and a carefully chosen h to ensure X is regular, enabling an exact reproduction of an auxiliary sequence y^* and thus effective re-expression of the original dynamical system. This work provides an explicit, algebraic bridge between discrete dynamical systems and RNNs with controlled stability guarantees, contributing to the understanding of how Lyapunov dynamics can be captured by recurrent architectures.

Abstract

We consider arbitrary bounded discrete time series originating from dynamical system with recursivity. More precisely, we provide an explicit construction of recurrent neural networks which effectively approximate the corresponding discrete dynamical systems.
Paper Structure (2 sections, 3 theorems, 25 equations)

This paper contains 2 sections, 3 theorems, 25 equations.

Table of Contents

  1. Introduction
  2. Proof of main theorem

Key Result

Theorem 1

Let $y:\mathbb{Z}\to[-1,1]$ be a time series generated by the discrete dynamical system in a delay coordinate $\Phi:[-1,1]^L\to[-1,1]^L$ ($Y_{t-1}\mapsto Y_t$) with recursivity and a finite $\lambda\geq 0$. Then for any large $K \in \mathbb{Z}_{\ge1}$ and for any constant $C$ which is slightly large This is only on the order of log worse (since $t=e^{\log t}$). Thus, this RNNs construction seems e

Theorems & Definitions (6)

  • Theorem 1
  • Proposition 2
  • proof
  • Lemma 3
  • proof
  • Remark 1