Evaluating the effectiveness of predicting covariates in LSTM Networks for Time Series Forecasting
Gareth Davies
TL;DR
This work investigates whether predicting covariates alongside a target variable with LSTM networks improves time-series forecasting. By introducing artificial, highly correlated covariates for future time steps across four public datasets and comparing a baseline LSTM to a segment-based variant, the study quantifies how covariate inclusion affects accuracy under varying horizon lengths and correlation levels. The findings show that covariates can yield improvements only in narrow, dataset-dependent conditions (e.g., very high $PCC$ and short horizons); in many cases, multivariate predictions perform worse than univariate forecasts, especially for longer horizons, though the segment-LSTM offers competitive performance when horizons are large. Overall, results suggest cautious use of covariate augmentation in LSTM forecasting and motivate exploring alternative architectures or real-world covariates for robust long-horizon predictions.
Abstract
Autoregressive Recurrent Neural Networks are widely employed in time-series forecasting tasks, demonstrating effectiveness in univariate and certain multivariate scenarios. However, their inherent structure does not readily accommodate the integration of future, time-dependent covariates. A proposed solution, outlined by Salinas et al 2019, suggests forecasting both covariates and the target variable in a multivariate framework. In this study, we conducted comprehensive tests on publicly available time-series datasets, artificially introducing highly correlated covariates to future time-step values. Our evaluation aimed to assess the performance of an LSTM network when considering these covariates and compare it against a univariate baseline. As part of this study we introduce a novel approach using seasonal time segments in combination with an RNN architecture, which is both simple and extremely effective over long forecast horizons with comparable performance to many state of the art architectures. Our findings from the results of more than 120 models reveal that under certain conditions jointly training covariates with target variables can improve overall performance of the model, but often there exists a significant performance disparity between multivariate and univariate predictions. Surprisingly, even when provided with covariates informing the network about future target values, multivariate predictions exhibited inferior performance. In essence, compelling the network to predict multiple values can prove detrimental to model performance, even in the presence of informative covariates. These results suggest that LSTM architectures may not be suitable for forecasting tasks where predicting covariates would typically be expected to enhance model accuracy.