Techniques for Enhancing Memory Capacity of Reservoir Computing

TL;DR

The paper addresses the memory capacity–nonlinearity trade-off in reservoir computing by reconfiguring input interfaces rather than changing reservoir dynamics. It introduces three methods—Delay, Pass through, and Clustering—that can be combined to boost memory retention, with the Delay method delivering the most significant gains. Using ESN and CBM-RC models on the NARMA benchmark and IPC analysis, the authors show that memory capacity can be increased while shaping the distribution of nonlinearity, and that the impact on IPC can be tuned via the number of delay steps. This approach provides a practical pathway to tailor RC systems for tasks requiring specific memory and nonlinear processing characteristics, including chip-friendly implementations.

Abstract

Reservoir Computing (RC) is a bio-inspired machine learning framework, and various models have been proposed. RC is a well-suited model for time series data processing, but there is a trade-off between memory capacity and nonlinearity. In this study, we propose methods to improve the memory capacity of reservoir models by modifying their network configuration except for the inside of reservoirs. The Delay method retains past inputs by adding delay node chains to the input layer with the specified number of delay steps. To suppress the effect of input value increase due to the Delay method, we divide the input weights by the number of added delay steps. The Pass through method feeds input values directly to the output layer. The Clustering method divides the input and reservoir nodes into multiple parts and integrates them at the output layer. We applied these methods to an echo state network (ESN), a typical RC model, and the chaotic Boltzmann machine (CBM)-RC, which can be efficiently implemented in integrated circuits. We evaluated their performance on the NARMA task, and measured information processing capacity (IPC) to evaluate the trade-off between memory capacity and nonlinearity.

Figures (4)

• Figure 1: Proposed methods.
• Figure 2: Examples of combining proposed methods.
• Figure 3: Evaluation results for NARMA tasks.
• Figure 4: Evaluation results for IPC.