Ensemble Reservoir Computing for Physical Systems

TL;DR

This work tackles the challenge of energy-efficient physical computing in the presence of intrinsic noise and temporal fluctuations by introducing Ensemble Reservoir Computing (ERC), which leverages ensemble averaging over spatially multiplexed, identical subsystems driven by a common input. The authors provide a theoretical framework proving that the ensemble-averaged observations $\,\mathbb{E}[\phi(x_k)]$ become time-invariant under certain conditions, effectively removing noise and fluctuations, and they extend ERC to exploit latent temporal dynamics via IPC and TIPC. They demonstrate ERC's effectiveness across diverse dynamical systems (including Lorenz, Rössler, Chua, COPY, and noisy ESNs) and, crucially, validate it on physical spin-torque oscillators (STOs), achieving up to 99.4% CRC accuracy and superior Hamming-code task performance compared to conventional RC. The results suggest ERC as a general, robust paradigm for harnessing time-variant physical substrates for reliable, high-performance computation with broad applicability to energy-efficient substrates and neuromorphic architectures.

Abstract

Physical computing exploits unconventional physical substrates to overcome limitations such as the high energy consumption inherent in digital computation. However, intrinsic noise and temporal fluctuations (e.g., oscillations) generally deteriorate computational performance. Here, we propose ensemble reservoir computing (ERC), a novel framework that employs ensemble averaging of spatially multiplexed systems to achieve robust information processing despite noise and temporal fluctuations. First, we prove that ensemble averaging in ERC eliminates temporal fluctuations and noise from dynamical states under certain conditions, thereby restoring computational performance to its noise-free level. Next, we show that ERC not only removes the noise and fluctuations but also actively exploits the computational capabilities that conventional reservoir computing (RC) leaves unutilized. This computational enhancement is demonstrated across diverse dynamical systems (e.g., periodic, chaotic, and strange-nonchaotic systems), in which ERC outperforms conventional RC. Finally, using energy-efficient spin-torque oscillators (STOs), we demonstrate that ERC maintains high performance even under realistic conditions, in which noise and temporal fluctuations coexist: STOs with ERC achieved 99\% accuracy on an error detection test, where conventional STO reservoir with linear regression only shows a chance level performance, highlighting ERC's robustness and performance gains for physical systems.

Figures (10)

• Figure S1: Schematics of ensemble reservoir computing (ERC).a) Examples of error factors $E$ in a physical system. In simulations, the physical reservoir state $\boldsymbol{x}$ treated as an ideal function without error factors $E$ (green). In real-world implementations, the dynamical state is observed as the measured state $\boldsymbol{z} = \phi(\boldsymbol{x})$, a nonlinear transformation of the original state $\boldsymbol{x}$ through $\phi$ that contains error factors $E$ (i.e., noise and temporal fluctuations; red). The green and red lines represent the ideal physical system and the observed state, respectively. b) Illustration of an ideal conventional RC scenario (upper) versus its practical limitations in the real world (lower). Conventional RC makes predictions using a linear sum of states that depends solely on input history. However, in practice, the error factors $E$ distort the observed states in physical systems. c) The operating principle of ERC. Parallel identical systems driven by a common input yield multiple observed states $\boldsymbol{z}_{(i)}~(i=1,2,\ldots,L)$. The ensemble average $\mathbb{E}[\phi(\boldsymbol{x})]$ over the observed states eliminates the error factors (i.e., noise and temporal fluctuations).
• Figure S2: Ensemble averaging restores and improves memory in time-variant systems.a) An example of noise removal using an echo state network (ESN). Three memory functions (MFs) and their memory capacities (MCs) are shown: a standard ESN (blue), an ESN with noise (black), and an ESN with ERC (red). To numerically verify Theorem \ref{['thm:thm1']}, we used $L=5\times10^5$ parallel ESNs with a network size $d=30$ and spectral radius $\rho=0.94$. b) Four examples that exploit temporal fluctuations: the Lorenz system, Rössler system, Chua circuit, and COPY map. The MFs obtained using conventional RC (black) and ERC (red) are illustrated. Their MCs under ERC are $1.08$ (Lorenz), $0.7$ (Rössler), $1.57$ (Chua), and $0.37$ (COPY).
• Figure S3: Application of ERC to a time-variant ESN. a) Prediction errors (NMSE) on the NARMA10 task as a function of $\rho$ for four configurations: a standard ESN without noise (blue), an ESN with additive noise (black), ERC with first-order ensemble average (red), and ERC with ensemble averages of powers $1$–$10$ (green). b) Information processing capacity (IPC) as a function of $\rho$ for the standard ESN (left) and ERC (right). The bars indicate the sum of $d$th-order IPCs (1st [blue], 2nd [green], and 3rd [red]), while the lines indicate the state rank.
• Figure S4: Application of ERC to a real spin-torque oscillator (STO). a) Schematic illustration of ERC applied to $L=60$ STOs (left). The spatially multiplexed STOs are illustrated using SEM images of the resist-pillar patterns before ion milling; these pillars serve as etching masks for the STO nanopillars. We inject the same binary input signal (upper right) to into all STOs and observe their resulting time series (lower right). b) The temporal information processing capacity (TIPC) of the original STO waveform and the IPC of the ensemble-averaged states using the observation functions $\phi\in\{x,x^2,\cos x,\mathrm{ReLU}(x),\mathrm{ADC}_{4}(x)\}$. The TIPC of the original waveform includes only time-variant capacities. In contrast, transformed states with several $\phi$ contain time-invariant components. Here, $x,x^2$, and $\cos x$ are theoretically admissible, while $\mathrm{ReLU}(x)$ and $\mathrm{ADC}_{4}(x)$ are not theoretically guaranteed. The color of each bar indicates the input order. The hatched and unhatched bars denote time-variant and time-invariant components, respectively. c) Time-series comparison among the target (black), non-averaged (blue), and ERC (red) signals for a cyclic redundancy check (CRC) task. We use $\phi\in\{x^2,x^4,x^6,x^8\}$ as observation functions to solve the CRC task.
• Figure S5: Finite-size comparison of the total retrievable MC.a) Some memory functions are illustrated: a standard ESN (blue) and an ESN with noise (black). The red line shows ERC results with $L=10$, $L=10^2$ (dash), and $L=10^5$ (solid). b) The MC as a function of the number of spatially multiplexed systems. The blue and black solid lines show the MC without noise and with noise, respectively. The red line shows the MC for spatially multiplexed systems with $L\in \{10,10^2,10^3,10^4,10^5\}$.

Theorems & Definitions (11)

• Theorem 1
• Theorem 6.1: Ensemble average transformation
• proof
• Corollary 6.2
• proof
• Definition 7.1
• Theorem 11.1: Desynchronization by noise
• proof
• Theorem 11.2: Elementary symmetric polynomials and even-powers ensemble
• proof