Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Universality of reservoir systems with recurrent neural networks

Hiroki Yasumoto, Toshiyuki Tanaka

TL;DR

Uniform strong universality of RNN reservoir systems for a certain class of dynamical systems is shown, which means that, given an approximation error to be achieved, one can construct an RNN reservoir system that approximates each target dynamical system in the class just via adjusting its linear readout.

Abstract

Approximation capability of reservoir systems whose reservoir is a recurrent neural network (RNN) is discussed. We show what we call uniform strong universality of RNN reservoir systems for a certain class of dynamical systems. This means that, given an approximation error to be achieved, one can construct an RNN reservoir system that approximates each target dynamical system in the class just via adjusting its linear readout. To show the universality, we construct an RNN reservoir system via parallel concatenation that has an upper bound of approximation error independent of each target in the class.

Universality of reservoir systems with recurrent neural networks

TL;DR

Uniform strong universality of RNN reservoir systems for a certain class of dynamical systems is shown, which means that, given an approximation error to be achieved, one can construct an RNN reservoir system that approximates each target dynamical system in the class just via adjusting its linear readout.

Abstract

Approximation capability of reservoir systems whose reservoir is a recurrent neural network (RNN) is discussed. We show what we call uniform strong universality of RNN reservoir systems for a certain class of dynamical systems. This means that, given an approximation error to be achieved, one can construct an RNN reservoir system that approximates each target dynamical system in the class just via adjusting its linear readout. To show the universality, we construct an RNN reservoir system via parallel concatenation that has an upper bound of approximation error independent of each target in the class.
Paper Structure (63 sections, 27 theorems, 60 equations, 3 figures)

This paper contains 63 sections, 27 theorems, 60 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Overview
  4. Notations
  5. Target dynamical systems
  6. Dynamical systems
  7. Assumptions for target dynamical systems
  8. Approximating reservoir systems
  9. Approximation error of a reservoir system for a dynamical system
  10. Overview
  11. Finite-length filters
  12. Norm of finite-length filters
  13. Approximation error
  14. Approximation bounds of feedforward neural networks
  15. A case of ReLU activation function
  16. ...and 48 more sections

Key Result

Proposition 11

There exists a universal constant $\kappa>0$ that has the following properties. Let $\mathcal{B}\subset\mathbb{R}^Q$ be a bounded set including the origin $\bm{0}$ and with non-empty interior. For any $M>0$, $g\in\mathcal{Z}_{M,\mathcal{B}}$, $N\in\mathbb{N}_+$, there exists an FNN with $4N$ hidden such that and such that

Figures (3)

  • Figure 1: Relations of coverings used in the proof.
  • Figure 2: Parallel concatenation of $J$ RNN reservoirs.
  • Figure 3: A target dynamical system and its order-$T$ cascaded dynamical system.

Theorems & Definitions (58)

  • Definition 1: Barron class Caragea_et_al:2023:barron_unif_improved
  • Remark 3
  • Remark 4
  • Remark 5
  • Remark 6
  • Definition 7
  • Definition 8
  • Definition 9: norm of finite-length filters
  • Definition 10: approximation error
  • Proposition 11: corrected and modified version of Proposition 2.2 in Caragea_et_al:2023:barron_unif_improved
  • ...and 48 more