Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Composite FORCE learning of chaotic echo state networks for time-series prediction

Yansong Li, Kai Hu, Kohei Nakajima, Yongping Pan

TL;DR

This paper tackles chaotic time-series prediction with echo-state networks (ESNs) trained online using FORCE learning. It introduces an RLS-based composite FORCE method that builds an extended regression via a stable filter $L(z)$ and leverages memory data $E(k)$ to relax persistency requirements and accelerate convergence. On Mackey-Glass data, the proposed method achieves faster parameter convergence, lower training and prediction errors ($\hat{W}_{\rm out}$ converges quickly and the MSEs drop to $0.0093$ and $0.0042$, respectively) compared with the basic RLS FORCE and LMS-based composite FORCE, while reducing the required reservoir size. This work enhances the robustness and efficiency of online chaotic time-series modeling and broadens the applicability of reservoir computing in real-time forecasting tasks.

Abstract

Echo state network (ESN), a kind of recurrent neural networks, consists of a fixed reservoir in which neurons are connected randomly and recursively and obtains the desired output only by training output connection weights. First-order reduced and controlled error (FORCE) learning is an online supervised training approach that can change the chaotic activity of ESNs into specified activity patterns. This paper proposes a composite FORCE learning method based on recursive least squares to train ESNs whose initial activity is spontaneously chaotic, where a composite learning technique featured by dynamic regressor extension and memory data exploitation is applied to enhance parameter convergence. The proposed method is applied to a benchmark problem about predicting chaotic time series generated by the Mackey-Glass system, and numerical results have shown that it significantly improves learning and prediction performances compared with existing methods.

Composite FORCE learning of chaotic echo state networks for time-series prediction

TL;DR

This paper tackles chaotic time-series prediction with echo-state networks (ESNs) trained online using FORCE learning. It introduces an RLS-based composite FORCE method that builds an extended regression via a stable filter and leverages memory data to relax persistency requirements and accelerate convergence. On Mackey-Glass data, the proposed method achieves faster parameter convergence, lower training and prediction errors ( converges quickly and the MSEs drop to and , respectively) compared with the basic RLS FORCE and LMS-based composite FORCE, while reducing the required reservoir size. This work enhances the robustness and efficiency of online chaotic time-series modeling and broadens the applicability of reservoir computing in real-time forecasting tasks.

Abstract

Echo state network (ESN), a kind of recurrent neural networks, consists of a fixed reservoir in which neurons are connected randomly and recursively and obtains the desired output only by training output connection weights. First-order reduced and controlled error (FORCE) learning is an online supervised training approach that can change the chaotic activity of ESNs into specified activity patterns. This paper proposes a composite FORCE learning method based on recursive least squares to train ESNs whose initial activity is spontaneously chaotic, where a composite learning technique featured by dynamic regressor extension and memory data exploitation is applied to enhance parameter convergence. The proposed method is applied to a benchmark problem about predicting chaotic time series generated by the Mackey-Glass system, and numerical results have shown that it significantly improves learning and prediction performances compared with existing methods.
Paper Structure (9 sections, 10 equations, 6 figures, 1 table)

This paper contains 9 sections, 10 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. The Proposed Method
  3. Network Architecture
  4. RLS-Based FORCE Learning
  5. RLS-Based Composite FORCE Learning
  6. Simulation Studies
  7. Numerical Setup
  8. Numerical Results
  9. Conclusions

Figures (6)

  • Figure 1: The ESN architecture used in this study, where only red arrows indicate trainable connections.
  • Figure 2: Chaotic time-series prediction by the ESN with the basic RLS-based FORCE learning. (a) Training performance, where only a period of the total time is displayed for a clear illustration. (b) Prediction performance, where only a period of the total time is displayed for a clear illustration. (c) The norm of the output weight $\hat{W}_{\rm out}$.
  • Figure 3: Chaotic time-series prediction by the ESN with the proposed RLS-based composite FORCE learning. (a) Training performance, where only a period of the total time is displayed for a clear illustration. (b) Prediction performance, where only a period of the total time is displayed for a clear illustration. (c) The norm of the output weight $\hat{W}_{\rm out}$.
  • Figure 4: Chaotic time-series prediction by the ESN with the LMS-based composite FORCE learning. (a) Training performance, where only a period of the total time is displayed for a clear illustration. (b) Prediction performance, where only a period of the total time is displayed for a clear illustration. (c) The norm of the output weight $\hat{W}_{\rm out}$.
  • Figure 5: Evolving trajectories of 10 randomly selected elements in the output weight $\hat{W}_{\rm out}$ under the three learning methods. (a) By the basic RLS-based FORCE learning. (b) By the proposed RLS-based composite FORCE learning. (c) By the LMS-based composite FORCE learning.
  • ...and 1 more figures