Asymptotic evaluation of the information processing capacity in reservoir computing

TL;DR

The paper tackles the problem of evaluating the information processing capacity (IPC) of reservoir computing (RC) in the limit of infinite data. It develops an asymptotic expansion for the training and test IPC, grounded in linear-regression theory, and introduces a weighted least-squares strategy to estimate the true IPC $C_0$ from finite data. It also provides robust criteria to detect zero IPC by examining the variance structure when $C_0=0$. Validation across a simple model, Legendre-polynomial tasks, and the NARMA10 benchmark demonstrates that the method closely tracks the infinite-data IPC and improves over naive long-data estimates, offering a more precise tool for RC performance evaluation, albeit with notable computational cost.

Abstract

Reservoir computing (RC) is becoming increasingly important because of its short training time. The squared error normalized by the target output is called the information processing capacity (IPC) and is used to evaluate the performance of an RC system. Since RC aims to learn the relationship between input and output time series, we should evaluate the IPC for infinitely long data rather than the IPC for finite-length data. However, a method for estimating it has not been established. We evaluated the IPC for infinitely long data using the asymptotic expansion of the IPC and weighted least-squares fitting. Then, we showed the validity of our method by numerical simulations. This work makes the performance evaluation of RC more evident.

Figures (4)

• Figure 1: The means and the variances of the training and the test IPC obtained by simulation in the simple system, and the theoretical lines obtained by the asymptotic expansions are plotted against $T$. Although the means of the training IPC fluctuate in the log-log plot, most samples are roughly on the theoretical lines, which indicates the effectiveness of the estimation.
• Figure 6: The means and the variances of the IPC using uniformly distributed input and Legendre first-order output were plotted, along with the asymptote estimated from them. The simulation results were mostly on the theoretical line, indicating the effectiveness of the estimation.
• Figure 11: The means and the variances of the IPC using uniformly distributed input and 15th-order Legendre output are plotted, along with the asymptote estimated from them. Although the means are roughly on the theoretical line, the variances are significantly off. Since the true IPC is nearly $0$, we can find that the variance decays faster than $1/T$. (In this graph, it decays at approximately $1/T^2$.)
• Figure 16: The means and the variances of the IPC obtained from the NARMA10 task and the asymptote estimated from them are plotted. The simulation results are almost on the theoretical lines.