Echo State Networks for Time Series Forecasting: Hyperparameter Sweep and Benchmarking

TL;DR

In the out-of-sample evaluation, the ESN performs on par with ARIMA and TBATS for monthly data and achieves the lowest mean MASE for quarterly data, while requiring lower computational cost than the more complex statistical models.

Abstract

This paper investigates the forecasting performance of Echo State Networks (ESNs) for univariate time series forecasting using a subset of the M4 Forecasting Competition dataset. Focusing on monthly and quarterly time series with at most 20 years of historical data, we evaluate whether a fully automatic, purely feedback-driven ESN can serve as a competitive alternative to widely used statistical forecasting methods. The study adopts a rigorous two-stage evaluation approach: a Parameter dataset is used to conduct an extensive hyperparameter sweep covering leakage rate, spectral radius, reservoir size, and information criteria for regularization, resulting in over four million ESN model fits; a disjoint Forecast dataset is then used for out-of-sample accuracy assessment. Forecast accuracy is measured using MASE and sMAPE and benchmarked against simple benchmarks like drift and seasonal naive and statistical models like ARIMA, ETS, and TBATS. The hyperparameter analysis reveals consistent and interpretable patterns, with monthly series favoring moderately persistent reservoirs and quarterly series favoring more contractive dynamics. Across both frequencies, high leakage rates are preferred, while optimal spectral radii and reservoir sizes vary with temporal resolution. In the out-of-sample evaluation, the ESN performs on par with ARIMA and TBATS for monthly data and achieves the lowest mean MASE for quarterly data, while requiring lower computational cost than the more complex statistical models. Overall, the results demonstrate that ESNs offer a compelling balance between predictive accuracy, robustness, and computational efficiency, positioning them as a practical option for automated time series forecasting.

Figures (4)

• Figure 1: Density plots illustrating the distributions of time series characteristics for the monthly (top row) and quarterly (bottom row) data. The left panels show the number of observations (series length), the middle panels display the strength of seasonality, and the right panels depict the strength of trend. The black lines represent the total M4 dataset, while the orange and blue lines correspond to the randomly sampled Parameter and Forecast datasets (Data source: M4 Forecasting Competition).
• Figure 2: Exemplary monthly time series with varying characteristics. The data are taken from the randomly sampled Parameter dataset and used to describe the ESN modeling framework (Data source: M4 Forecasting Competition).
• Figure 3: Forecasts for the example monthly series M21655 under varying hyperparameter settings. Left: forecasts for leakage rates $\alpha \in \{0.1, 0.2, ..., 0.9, 1.0\}$. Right: forecasts for spectral radii $\rho \in \{0.2, 0.3, ..., 1.1, 1.2\}$. Actual observations (hold-out data) are shown in black. Colored forecast lines transition from blue (low values) to orange (high values), illustrating how increasing $\alpha$ and $\rho$ alters forecast smoothness, reactivity, and temporal lag.
• Figure 4: Median MASE across all time series as a function of individual hyperparameter settings, computed by marginalizing over all remaining grid configurations. Monthly results are shown in orange and quarterly results in blue. Each point denotes the median MASE for a specific hyperparameter value; the best-performing value per panel (minimum median MASE) is indicated by a filled marker, while all other values are shown as hollow markers.