Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Feature-Based Echo-State Networks: A Step Towards Interpretability and Minimalism in Reservoir Computer

Debdipta Goswami

TL;DR

The paper tackles the interpretability and efficiency gap of standard echo-state networks (ESNs) for time-series prediction of dynamical systems. It introduces Feature-Based Echo-State Networks (Feat-ESN), which deploy parallel small reservoirs driven by input feature subsets and combine them with a nonlinear readout to enable per-feature interpretability and reduced reservoir size. Key contributions include a scalable, interpretable reservoir architecture, a delay-embedding extension for partial observations, and empirical validation on chaotic systems and real traffic data showing improved accuracy with fewer nodes. This approach offers a practically efficient and interpretable reservoir computing framework suitable for high-dimensional and partially observed time-series tasks such as traffic forecasting.

Abstract

This paper proposes a novel and interpretable recurrent neural-network structure using the echo-state network (ESN) paradigm for time-series prediction. While the traditional ESNs perform well for dynamical systems prediction, it needs a large dynamic reservoir with increased computational complexity. It also lacks interpretability to discern contributions from different input combinations to the output. Here, a systematic reservoir architecture is developed using smaller parallel reservoirs driven by different input combinations, known as features, and then they are nonlinearly combined to produce the output. The resultant feature-based ESN (Feat-ESN) outperforms the traditional single-reservoir ESN with less reservoir nodes. The predictive capability of the proposed architecture is demonstrated on three systems: two synthetic datasets from chaotic dynamical systems and a set of real-time traffic data.

Feature-Based Echo-State Networks: A Step Towards Interpretability and Minimalism in Reservoir Computer

TL;DR

The paper tackles the interpretability and efficiency gap of standard echo-state networks (ESNs) for time-series prediction of dynamical systems. It introduces Feature-Based Echo-State Networks (Feat-ESN), which deploy parallel small reservoirs driven by input feature subsets and combine them with a nonlinear readout to enable per-feature interpretability and reduced reservoir size. Key contributions include a scalable, interpretable reservoir architecture, a delay-embedding extension for partial observations, and empirical validation on chaotic systems and real traffic data showing improved accuracy with fewer nodes. This approach offers a practically efficient and interpretable reservoir computing framework suitable for high-dimensional and partially observed time-series tasks such as traffic forecasting.

Abstract

This paper proposes a novel and interpretable recurrent neural-network structure using the echo-state network (ESN) paradigm for time-series prediction. While the traditional ESNs perform well for dynamical systems prediction, it needs a large dynamic reservoir with increased computational complexity. It also lacks interpretability to discern contributions from different input combinations to the output. Here, a systematic reservoir architecture is developed using smaller parallel reservoirs driven by different input combinations, known as features, and then they are nonlinearly combined to produce the output. The resultant feature-based ESN (Feat-ESN) outperforms the traditional single-reservoir ESN with less reservoir nodes. The predictive capability of the proposed architecture is demonstrated on three systems: two synthetic datasets from chaotic dynamical systems and a set of real-time traffic data.
Paper Structure (11 sections, 1 theorem, 11 equations, 12 figures, 1 table, 1 algorithm)

This paper contains 11 sections, 1 theorem, 11 equations, 12 figures, 1 table, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. Echo-State Networks for Dynamical Systems Prediction
  3. Feature-Based Echo-State Network: A Bite Sized Approach
  4. Input map
  5. Reservoirs
  6. Readout
  7. Numerical Examples
  8. Lorenz System
  9. Rössler System
  10. Prediction of Traffic Volume on an Intersection of a Road Network
  11. Conclusion

Key Result

Lemma 1

If the feature matrix $W_f$ has the full column rank, $m<N_fb$, and the readout functions are chosen from a subalgebra of $C(\mathbb{R}^n,\mathbb{R}^n)$ that separate points, then the Feat-ESN maps $\{\mathbf{u}(t_k)\}_k \mapsto \{\mathbf{y}(t_k)\}_k$ are dense in $C(\mathbb{U},\mathbb{Y})$ where $\

Figures (12)

  • Figure 1: Architecture and training of an ESN: (a) the basic ESN, (b) Feature-based ESN (Feat-ESN)
  • Figure 2: prediction of the noisy time-series $x(t_k)$ and $z(t_k)$ from Lorenz system \ref{['Eq: Lorenz']} with $b=100$ and $N_f=7$, i.e., reservoir size $n=700$: (a) true and predicted signal with Feat-ESN, (b) true and predicted signal with regular ESN
  • Figure 3: Error profile of Lorenz time-series prediction: NRMSE with different block-size $b$
  • Figure 4: Frobenius norm of the output map for different features of Lorenz time-series prediction
  • Figure 5: prediction of the noisy time-series $x(t_k)$ and $z(t_k)$ from Rössler system \ref{['Eq: Rossler']} with $b=100$ and $N_f=7$, i.e., reservoir size $n=700$: (a) true and predicted signal with Feat-ESN, (b) true and predicted signal with least square training
  • ...and 7 more figures

Theorems & Definitions (5)

  • Remark 1
  • Remark 2
  • Remark 3
  • Lemma 1
  • proof