Storage and selection of multiple chaotic attractors in minimal reservoir computers

Abstract

Modern predictive modeling increasingly calls for a single learned dynamical substrate to operate across multiple regimes. From a dynamical-systems viewpoint, this capability decomposes into the storage of multiple attractors and the selection of the appropriate attractor in response to contextual cues. In reservoir computing (RC), multi-attractor learning has largely been pursued using large, randomly wired reservoirs, on the assumption that stochastic connectivity is required to generate sufficiently rich internal dynamics. At the same time, recent work shows that minimal deterministic reservoirs can match random designs for single-system chaotic forecasting. Under which conditions can minimal topologies learn multiple chaotic attractors? In this paper, we find that minimal architectures can successfully store multiple chaotic attractors. However, these same architectures struggle with task switching, in which the system must transition between attractors in response to external cues. We test storage and selection on all 28 unordered system pairs formed from eight three-dimensional chaotic systems. We do not observe a robust dependence of multi-attractor performance on reservoir topology. Over the ten topologies investigated, we find that no single one consistently outperforms the others for either storage or cue-dependent selection. Our results suggest that while minimal substrates possess the representational capacity to model coexisting attractors, they may lack the robust temporal memory required for cued transitions.

Figures (8)

• Figure 1: Minimal deterministic reservoir topologies. Each panel shows the nonzero entries (magenta) of the $10\times10$ reservoir matrix $\mathbf{W}$. All the nonzero weights have the same magnitude and sign. Zero entries are shown in light gray. Panels correspond to the following structures: (a) delay line (DL), (b) delay line with feedback connections (DLFB), (c) simple cycle (SC), (d) cycle with jumps (CJ), (e) self-loop cycle (SLC), (f) self-loop feedback cycle (SLFB), (g) self-loop delay line with backward connections (SLDB), (h) self-loop with forward connections (SLFC), (i) forward connections (FC), and (j) double cycle (DC).
• Figure 2: Approaches for multi-attractor learning in reservoir computing. Panel (a) shows the approach for attractor storage in an echo state network (ESN) using the blending technique (BT). Two chaotic trajectories are provided in parallel by concatenating their state vectors into a single input, $\mathbf{u}_{\mathrm{bt}}(t)=[\mathbf{u}_{\chi_1}(t);\mathbf{u}_{\chi_2}(t)]$. This joint input drives the ESN, and a single linear readout is trained to produce a concatenated output trajectory that reconstructs both systems simultaneously (right time series and attractors; colors distinguish the two systems). Panel (b) illustrates cue-dependent selection using the parameter-aware (PA) approach. In addition to the state input, a scalar cue $\beta_i$ identifying the target attractor is injected through the bias term in Eq. \ref{['mmfrc:eq:paesn']}. The ESN is then run with $\beta_i$ held fixed so that the autonomous dynamics are expected to converge to, and reproduce, the attractor associated with that cue. In both panels, the light pink boxes indicate untrained matrices, while the light violet box indicates the trained readout.
• Figure 3: Effect of prediction deterioration on metric changes. In this figure, we provide Lorenz attractor overlays, illustrating how reconstruction quality maps to the discrete state--space Kullback--Leibler divergence (KLD). Grey curves show the reference Lorenz trajectory. In panel (a), the comparison trajectory (pink) is generated by the same Lorenz equations but using different initial conditions; both the standard discrete KLD (dKLD) and the penalized variant (KLD) remain small. Panels (b--e) show progressively degraded ESN forecasts (violet). As prediction quality deteriorates and forecasted trajectories increasingly deviate from the true state space, the standard dKLD exhibits relatively low sensitivity. On the other hand, the penalized KLD increases with the degree of prediction failure, providing a clearer indication of reconstruction failure. Metric values reported below each panel are in $\log_{10}$ scale.
• Figure 4: Multi-attractor learning accuracy across minimal reservoir topologies. Panel (a) shows the results of the blending technique (BT). The violin plots indicate the distributions of $\log_{10}$ Kullback-Leibler divergence (KLD) over all 28 unordered system pairs for each reservoir topology (ordered by increasing median). Panel (b) illustrates the results for the parameter-aware (PA) approach. The panel maintains the same topology ordering and visualization used in BT. In both panels, violins show the distribution across system pairs, and the black dot marks the median $\log_{10}\mathrm{KLD}$ for that topology; the apparent extension below zero is due to violin smoothing and does not imply negative KLD values. (c) Representative system pair reconstruction example for Lorenz–SprottS using the self-loop delay line with backward connections (SLDB) topology in the BT setting. Dashed curves show the reference attractors and solid curves the ESN-generated attractors, with different colors distinguishing the two systems. The accuracy for the system pair corresponds to $\log_{10}\mathrm{KLD}=0.692$.
• Figure 5: Performance of minimal reservoirs across multi-attractor learning tasks and system pairs. (a) Heat map of prediction quality for each combination of reservoir initializer and chaotic system pair under the blending technique (BT) approach, quantified as the log$_{10}$ of the mean KLD over runs. Panel (b) reports the same results as Panel (a), but for the parameter-aware (PA) training scheme. In both panels, darker tiles indicate lower error (better forecasts), indicating how individual minimal initializers generalize across different blended systems and how their performance patterns change between training schemes.