An Open-Source and Reproducible Implementation of LSTM and GRU Networks for Time Series Forecasting

TL;DR

The paper addresses the need for reproducible open-source baselines in time-series forecasting with LSTM and GRU networks. It presents an end-to-end implementation and evaluation on two datasets—a real financial BANKEX series and a synthetic Activities series—using RMSE and DA to quantify predictive accuracy and directional correctness. Key findings show that LSTM and GRU substantially improve forecasts over a naive baseline on the Activities dataset, while improvements on BANKEX are not evident, highlighting data-dependent performance and the importance of hyperparameter tuning. The work contributes an openly available, reproducible framework and datasets to enable future benchmarking and comparisons across forecasting methods, with implications for practitioners seeking transparent evaluation in time-series forecasting.

Abstract

This paper introduces an open-source and reproducible implementation of Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) Networks for time series forecasting. We evaluated LSTM and GRU networks because of their performance reported in related work. We describe our method and its results on two datasets. The first dataset is the S&P BSE BANKEX, composed of stock time series (closing prices) of ten financial institutions. The second dataset, called Activities, comprises ten synthetic time series resembling weekly activities with five days of high activity and two days of low activity. We report Root Mean Squared Error (RMSE) between actual and predicted values, as well as Directional Accuracy (DA). We show that a single time series from a dataset can be used to adequately train the networks if the sequences in the dataset contain patterns that repeat, even with certain variation, and are properly processed. For 1-step ahead and 20-step ahead forecasts, LSTM and GRU networks significantly outperform a baseline on the Activities dataset. The baseline simply repeats the last available value. On the stock market dataset, the networks perform just like the baseline, possibly due to the nature of these series. We release the datasets used as well as the implementation with all experiments performed to enable future comparisons and to make our research reproducible.

Figures (5)

• Figure 1: LSTM (left) and GRU (right). $c$ represents the memory cell and $\tilde{c}$ the new memory cell of the LSTM. $h$ represents the activation and $\tilde{h}$ the new activation of the GRU. Based on chung2014empirical.
• Figure 2: (a) Time series in the BANKEX dataset without normalization. Closing Price in Indian Rupee (INR). Daily samples retrieved between July 12, 2005 and November 3, 2017 using Yahoo! Finance's API YahooF_API. All time series with $3\,032$ samples. (b) Same time series as in (a). Closing Price normalized between 0 and 1. The numbers from 1 to 10 correspond to the numbers (first column) for each series in Table \ref{['tab1']}.
• Figure 3: Time series in the Activities dataset without normalization, first 100 samples.
• Figure 4: Examples of 20-step ahead forecast.
• Figure 5: Example of 1-step ahead forecast. Actual and predicted closing price over 251 days, test set Yes Bank. Closing Price in Indian Rupee (INR).