Linear Simple Cycle Reservoirs at the edge of stability perform Fourier decomposition of the input driving signals

TL;DR

The paper investigates the representational structure of linear Simple Cycle Reservoirs (SCR) at the edge of stability ($\rho=1$) through a kernel perspective on reservoir dynamics. By treating SCR state space as a feature map and analyzing the induced time-series kernel via its metric tensor $Q$, it proves that SCR motifs converge to the Fourier basis in both complex and real domains, depending on the dynamical coupling being unitary or orthogonal. Key contributions include a theoretical derivation showing $Q$ diagonalizes in the Fourier basis for $\rho=1$, a characterization of motif counts in the real domain ($\lceil n/2\rceil$ symmetric and $\lfloor n/2\rfloor$ skew-symmetric motifs), and an explicit construction of a real Fourier motif matrix. Numerical experiments with Reservoir Motif Machines (RMM) corroborate that SCR motifs at $\rho=1$ and Fourier motifs yield essentially identical feature representations for univariate time-series forecasting, highlighting a deep connection between RC kernels and classical Fourier analysis. This work clarifies why motif richness collapses at the edge of stability and suggests practical ways to leverage Fourier-like representations in RC-based signal processing and forecasting tasks.

Abstract

This paper explores the representational structure of linear Simple Cycle Reservoirs (SCR) operating at the edge of stability. We view SCR as providing in their state space feature representations of the input-driving time series. By endowing the state space with the canonical dot-product, we ``reverse engineer" the corresponding kernel (inner product) operating in the original time series space. The action of this time-series kernel is fully characterized by the eigenspace of the corresponding metric tensor. We demonstrate that when linear SCRs are constructed at the edge of stability, the eigenvectors of the time-series kernel align with the Fourier basis. This theoretical insight is supported by numerical experiments.

Figures (7)

• Figure 1: Example of Fourier coefficient of linear SCR at $\rho = 0.9812798473475446, 0.999, 1$ respectively. $0.9812798473475446$ in particular is where the relative area peaks in Figure \ref{['fig:relative_area']}.
• Figure 2: Relative area of Linear SCR and Randomly generated Reservoir with respect to the spectral radius
• Figure 3: Column-wise FFT of motifs of linear SCR with $\rho = 1$ and the column-wise FFT of $\hbox{\bf {F}}$. The first row shows the Fourier spectra in the original form and the second row has their columns rearranged with the same shuffling indices.
• Figure 4: Example of 4 randomly chosen motifs and their corresponding Fourier basis. Notice some are off by a phase of $\pi$.
• Figure 5: Plots of Fourier basis number $93$ constructed under Equations \ref{['eqn:fourier']} sampled under two different frequencies.

Theorems & Definitions (22)

• Definition 2.1
• Definition 2.2
• Definition 2.3
• Remark 2.4
• Lemma 4.1
• proof
• Corollary 4.2
• proof
• Lemma 4.3
• proof