Structuring Multiple Simple Cycle Reservoirs with Particle Swarm Optimization

TL;DR

This work introduces Multiple Simple Cycle Reservoirs (MSCR), a multi-reservoir RC framework built from SCR vertex-encoders connected via a weighted DAG. By applying Particle Swarm Optimization (PSO) to jointly tune input scalings and the inter-reservoir topology (encoded in ${\bf H}$, ${\bf d}$, and ${\bf A}$), MSCR-PSO achieves competitive predictive accuracy with lower state dimensionality across three benchmark time-series datasets, outperforming GA-based MSCR methods and, in several cases, single SCR baselines. The results demonstrate the flexibility of MSCR to adapt topology to task structure, including rank-one reductions that effectively collapse to a single SCR when optimal. The approach offers a promising, hardware-friendly path for efficient AI devices by enabling continuous optimization of both node-level gains and network connectivity without excessive state-space growth.

Abstract

Reservoir Computing (RC) is a time-efficient computational paradigm derived from Recurrent Neural Networks (RNNs). The Simple Cycle Reservoir (SCR) is an RC model that stands out for its minimalistic design, offering extremely low construction complexity and proven capability of universally approximating time-invariant causal fading memory filters, even in the linear dynamics regime. This paper introduces Multiple Simple Cycle Reservoirs (MSCRs), a multi-reservoir framework that extends Echo State Networks (ESNs) by replacing a single large reservoir with multiple interconnected SCRs. We demonstrate that optimizing MSCR using Particle Swarm Optimization (PSO) outperforms existing multi-reservoir models, achieving competitive predictive performance with a lower-dimensional state space. By modeling interconnections as a weighted Directed Acyclic Graph (DAG), our approach enables flexible, task-specific network topology adaptation. Numerical simulations on three benchmark time-series prediction tasks confirm these advantages over rival algorithms. These findings highlight the potential of MSCR-PSO as a promising framework for optimizing multi-reservoir systems, providing a foundation for further advancements and applications of interconnected SCRs for developing efficient AI devices.

Figures (4)

• Figure 1: A schematic diagram of an MSCR of order three for dealing with time-series prediction tasks. In the shown case, all the vertex-encoder are valid, and the corresponding rank of the MSCR is three.
• Figure 2: Glimpses of three time-series datasets: MG-17 (left), NARMA-10 (middle), MSTSN (right).
• Figure 3: The best average prediction performances of all tested models on three datasets. (a) MG-17, (b) NARMA-10, and (c) MSTSN.
• Figure 4: Visualization of searched network topologies (controlled by $\mathbf{d}$ and $\mathbf{A}$) and input scalings (controlled by $\mathbf{s}$ and $\mathbf{H}$) corresponding to the given prediction tasks: MG-17 (left), NARMA-10 (middle), and MSTSN (right).

Theorems & Definitions (6)

• Definition 2.1
• Definition 2.2
• Definition 2.3
• Definition 2.4
• Definition 2.5
• Remark 2.6