Asymmetrically connected reservoir networks learn better

TL;DR

The paper investigates how reservoir-network topology, specifically symmetry and connectivity structure, governs reservoir computing performance. Using the Mackey–Glass time-series as a benchmark, it shows that completely random, asymmetric reservoirs outperform all symmetric or structured reservoirs, including small-world topologies, in both open- and closed-loop settings. The authors quantify this with mean-squared error, valid prediction time, and a task-independent information processing capacity (IPC), finding the highest IPC total for asymmetric random reservoirs. The results argue that maximizing structural disorder enhances computational power and align with biological network characteristics, with implications for energy-efficient RC design and future exploration of binary connections and more complex models.

Abstract

We show that connectivity within the high-dimensional recurrent layer of a reservoir network is crucial for its performance. To this end, we systematically investigate the impact of network connectivity on its performance, i.e., we examine the symmetry and structure of the reservoir in relation to its computational power. Reservoirs with random and asymmetric connections are found to perform better for an exemplary Mackey-Glass time series than all structured reservoirs, including biologically inspired connectivities, such as small-world topologies. This result is quantified by the information processing capacity of the different network topologies which becomes highest for asymmetric and randomly connected networks.

Figures (4)

• Figure 1: Representative configurations in a simple network with four nodes. The first column shows the reservoir matrix with color-coded weights. The second column shows the network graphs with color-coded weights. The thicker the arrow the higher weight magnitude. The third column displays the node degree distributions $P(k)$ with $N_r=1024$ nodes plotting incoming degrees (in) and outgoing degrees (out). Rows 1 to 5 from top to bottom show random--asymmetric (R-A), random symmetric--asymmetric (RS-A), random symmetric--symmetric (RS-S), Watts-Strogatz--asymmetric (WS-A) with $p=1$, and Watts-Strogatz--symmetric (WS-S) with $p=1$. The first part of the acronym stands for the connectivity matrix $A$, the second one for the weight matrix $W^c$, see also eq. \ref{['eq:res']}. Color bar holds for all cases.
• Figure 2: Open-loop reservoir performance as sketched in the central upper region. Double logarithmic plot of the median of the Mean-Square-Error (MSE) for all 5 reservoirs, for WS-A, and WS-S cases also dependent on rewiring probability $p$. The error bars stand for the median absolute deviation (MAD), which is computed over an ensemble of $100$ differently initialized reservoir networks for each of the 5 configurations.
• Figure 3: Closed-loop reservoir performance as sketched in the central upper region of the top panel. Top: Double logarithmic plot of the median of the Mean-Square-Error (MSE) from all network configurations similar to Fig. \ref{['fig:open-loop']}. Bottom: Median of the valid prediction time $T_{\rm vp}$ in units of Lyapunov time $1/\lambda_1$ for all 5 network configurations, for WS-A, and WS-S additionally as a function of $p$. The error bars stand now for the median absolute deviation (MAD), which is computed over an ensemble of $100$ differently initialized reservoirs for each specific network configuration.
• Figure 4: Information processing capacities of degree $d$ and accumulated total information processing capacity $\hbox{IPC}_{\rm total}$ for upper degree bound $D=5$. The median of total information processing capacities IPC$_{\rm total}$ (blue squares) and their decomposition with respect to the degree $d$ (see legend) for the 5 reservoir configurations. The error bars represent the median absolute deviation, which is computed over an ensemble of $40$ differently initialized networks for each specific network configuration. The solid line shows the upper bound of $\rm IPC_{total}\le N_r =1024$ for the reservoir configurations.