Optimizing the Network Topology of a Linear Reservoir Computer

TL;DR

This work tackles the design of reservoir computer connectivity by constraining to a linear reservoir and decoupling its dynamics into independent modes via a modal decomposition. It then optimizes each mode by selecting eigenvalues of the adjacency matrix, formulated in the frequency domain to reduce the readout problem size. The proposed objective combines frequency-domain training error, regularization on output weights, and a barrier that encourages diverse eigenvalues, solved with nonlinear optimization and warm-starts. Numerical results show the optimized linear RC outperforms randomly connected reservoirs and often matches or surpasses nonlinear RCs of similar size, offering practical performance gains with enhanced interpretability and scalability.

Abstract

Machine learning has become a fundamental approach for modeling, prediction, and control, enabling systems to learn from data and perform complex tasks. Reservoir computing is a machine learning tool that leverages high-dimensional dynamical systems to efficiently process temporal data for prediction and observation tasks. Traditionally, the connectivity of the network that underlies a reservoir computer (RC) is generated randomly, lacking a principled design. Here, we focus on optimizing the connectivity of a linear RC to improve its performance and interpretability, which we achieve by decoupling the RC dynamics into a number of independent modes. We then proceed to optimize each one of these modes to perform a given task, which corresponds to selecting an optimal RC connectivity in terms of a given set of eigenvalues of the RC adjacency matrix. Simulations on networks of varying sizes show that the optimized RC significantly outperforms randomly constructed reservoirs in both training and testing phases and often surpasses nonlinear reservoirs of comparable size. This approach provides both practical performance advantages and theoretical guidelines for designing efficient, task-specific, and analytically transparent RC architectures.

Figures (6)

• Figure 1: Two equivalent RC network topologies. a) A coupled reservoir computer. b) A decoupled reservoir computer.
• Figure 2: Performance of the 10-node reservoir. Red dashed curves represent the training signal, while blue curves correspond to the best fit produced by the reservoir: (a) Training before optimization, (b) Testing before optimization, (c) Training after optimization, (d) Testing after optimization.
• Figure 3: Performance of the 100-node reservoir. Red dashed curves represent the training signal, while blue curves correspond to the best fit produced by the reservoir: (a) Training before optimization, (b) Testing before optimization, (c) Training after optimization, (d) Testing after optimization.
• Figure 4: Performance of an $N=25$-node reservoir used to observe the dynamics of the chaotic Lorenz system. Both the input and training signals are restricted to the $K=6$ dominant common frequencies. Red dashed curves represent the training signal, while blue curves correspond to the best fit produced by the reservoir: (a) Training before optimization, (b) Testing before optimization, (c) Training after optimization, (d) Testing after optimization.
• Figure 5: The figures show the average normalized root mean squared testing error (NRMSE) of reservoir computers with their corresponding standard deviations as we vary either the number of input frequencies $K$ or the number of nodes $N$. Mean and standard deviations are computed from 20 independent runs. The yellow curve represents the NRMSE of the linear benchmark, the orange curve corresponds to the nonlinear RC with ReLU activation, the blue curve shows the nonlinear RC with tanh activation, and the purple curve indicates the optimized LRC. a) The number of nodes is fixed at $N=50$ while the number of input frequencies varies. b) The number of input frequencies is fixed at $K=10$ while the number of reservoir nodes varies. The duration of the testing phase is taken to be one third of the duration of the training phase.

Theorems & Definitions (4)

• Theorem 1
• proof : Proof
• Theorem 2
• proof : Proof