Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Predictive Modeling in the Reservoir Kernel Motif Space

Peter Tino, Robert Simon Fong, Roberto Fabio Leonarduzzi

TL;DR

This work introduces a kernel-based view of linear reservoirs and proposes the Reservoir Motif Machine (RMM), which represents recent time-series blocks in a motif space derived from the reservoir kernel and predicts via a linear readout. By learning motif-weighted projections of histories, Lin-RMM achieves strong univariate forecasting performance, often surpassing transformer-based methods, and remains competitive on multivariate tasks where deep models excel. The authors provide a geometric interpretation, connect motif axes to the reservoir Gram matrix, and demonstrate that simple, memory-rich representations can outperform more complex architectures on several benchmarks. The findings emphasize the value of principled, memory-aware baselines and offer interpretable motifs as a lens into temporal structure in time-series data.

Abstract

This work proposes a time series prediction method based on the kernel view of linear reservoirs. In particular, the time series motifs of the reservoir kernel are used as representational basis on which general readouts are constructed. We provide a geometric interpretation of our approach shedding light on how our approach is related to the core reservoir models and in what way the two approaches differ. Empirical experiments then compare predictive performances of our suggested model with those of recent state-of-art transformer based models, as well as the established recurrent network model - LSTM. The experiments are performed on both univariate and multivariate time series and with a variety of prediction horizons. Rather surprisingly we show that even when linear readout is employed, our method has the capacity to outperform transformer models on univariate time series and attain competitive results on multivariate benchmark datasets. We conclude that simple models with easily controllable capacity but capturing enough memory and subsequence structure can outperform potentially over-complicated deep learning models. This does not mean that reservoir motif based models are preferable to other more complex alternatives - rather, when introducing a new complex time series model one should employ as a sanity check simple, but potentially powerful alternatives/baselines such as reservoir models or the models introduced here.

Predictive Modeling in the Reservoir Kernel Motif Space

TL;DR

This work introduces a kernel-based view of linear reservoirs and proposes the Reservoir Motif Machine (RMM), which represents recent time-series blocks in a motif space derived from the reservoir kernel and predicts via a linear readout. By learning motif-weighted projections of histories, Lin-RMM achieves strong univariate forecasting performance, often surpassing transformer-based methods, and remains competitive on multivariate tasks where deep models excel. The authors provide a geometric interpretation, connect motif axes to the reservoir Gram matrix, and demonstrate that simple, memory-rich representations can outperform more complex architectures on several benchmarks. The findings emphasize the value of principled, memory-aware baselines and offer interpretable motifs as a lens into temporal structure in time-series data.

Abstract

This work proposes a time series prediction method based on the kernel view of linear reservoirs. In particular, the time series motifs of the reservoir kernel are used as representational basis on which general readouts are constructed. We provide a geometric interpretation of our approach shedding light on how our approach is related to the core reservoir models and in what way the two approaches differ. Empirical experiments then compare predictive performances of our suggested model with those of recent state-of-art transformer based models, as well as the established recurrent network model - LSTM. The experiments are performed on both univariate and multivariate time series and with a variety of prediction horizons. Rather surprisingly we show that even when linear readout is employed, our method has the capacity to outperform transformer models on univariate time series and attain competitive results on multivariate benchmark datasets. We conclude that simple models with easily controllable capacity but capturing enough memory and subsequence structure can outperform potentially over-complicated deep learning models. This does not mean that reservoir motif based models are preferable to other more complex alternatives - rather, when introducing a new complex time series model one should employ as a sanity check simple, but potentially powerful alternatives/baselines such as reservoir models or the models introduced here.
Paper Structure (17 sections, 22 equations, 4 figures, 4 tables)

This paper contains 17 sections, 22 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Linear reservoir as a temporal kernel
  3. Predictive model based on reservoir motifs
  4. Geometric view
  5. Experiments
  6. Datasets
  7. ETT
  8. ECL
  9. Weather
  10. Exchange
  11. ILI
  12. Experimental setup
  13. Time series prediction
  14. Univariate prediction
  15. Multivariate prediction
  16. ...and 2 more sections

Figures (4)

  • Figure 1: Six most relevant motifs for the electricity consumption prediction (ECL dataset) - 48h (2 days) prediction horizon.
  • Figure 2: Six most relevant motifs for the electricity consumption prediction (ECL dataset) - 168h (7 days) prediction horizon.
  • Figure 3: Six most relevant motifs for the electricity transformer oil temperature prediction (ETTh dataset) - 48h (2 days) prediction horizon, region 1.
  • Figure 4: Six most relevant motifs for the electricity transformer oil temperature prediction (ETTh dataset) - 48h (2 days) prediction horizon, region 2.