Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

What Matters in Deep Learning for Time Series Forecasting?

Valentina Moretti, Andrea Cini, Ivan Marisca, Cesare Alippi

TL;DR

This paper examines why deep learning advances in time series forecasting are difficult to interpret, arguing that four design dimensions—model configuration (global/local/hybrid), preprocessing and exogenous inputs, temporal processing, and spatial processing—shape reported performance. Through extensive experiments on four benchmarks (Electricity, Weather, Traffic, Solar), it shows that simple, well-founded architectures can match state-of-the-art results, and that many gains can be traced to overlooked implementation details rather than novel operators. The authors critique current benchmarking practices for conflating design choices with architectural advances, and advocate a dimension-aware evaluation framework plus an auxiliary Forecasting Model Card to improve transparency and reproducibility. They conclude that grounding forecasting models in principled design for groups of time series is crucial for reliable progress and propose concrete steps to mature benchmarking in the field.

Abstract

Deep learning models have grown increasingly popular in time series applications. However, the large quantity of newly proposed architectures, together with often contradictory empirical results, makes it difficult to assess which components contribute significantly to final performance. We aim to make sense of the current design space of deep learning architectures for time series forecasting by discussing the design dimensions and trade-offs that can explain, often unexpected, observed results. This paper discusses the necessity of grounding model design on principles for forecasting groups of time series and how such principles can be applied to current models. In particular, we assess how concepts such as locality and globality apply to recent forecasting architectures. We show that accounting for these aspects can be more relevant for achieving accurate results than adopting specific sequence modeling layers and that simple, well-designed forecasting architectures can often match the state of the art. We discuss how overlooked implementation details in existing architectures (1) fundamentally change the class of the resulting forecasting method and (2) drastically affect the observed empirical results. Our results call for rethinking current faulty benchmarking practices and the need to focus on the foundational aspects of the forecasting problem when designing architectures. As a step in this direction, we propose an auxiliary forecasting model card, whose fields serve to characterize existing and new forecasting architectures based on key design choices.

What Matters in Deep Learning for Time Series Forecasting?

TL;DR

This paper examines why deep learning advances in time series forecasting are difficult to interpret, arguing that four design dimensions—model configuration (global/local/hybrid), preprocessing and exogenous inputs, temporal processing, and spatial processing—shape reported performance. Through extensive experiments on four benchmarks (Electricity, Weather, Traffic, Solar), it shows that simple, well-founded architectures can match state-of-the-art results, and that many gains can be traced to overlooked implementation details rather than novel operators. The authors critique current benchmarking practices for conflating design choices with architectural advances, and advocate a dimension-aware evaluation framework plus an auxiliary Forecasting Model Card to improve transparency and reproducibility. They conclude that grounding forecasting models in principled design for groups of time series is crucial for reliable progress and propose concrete steps to mature benchmarking in the field.

Abstract

Deep learning models have grown increasingly popular in time series applications. However, the large quantity of newly proposed architectures, together with often contradictory empirical results, makes it difficult to assess which components contribute significantly to final performance. We aim to make sense of the current design space of deep learning architectures for time series forecasting by discussing the design dimensions and trade-offs that can explain, often unexpected, observed results. This paper discusses the necessity of grounding model design on principles for forecasting groups of time series and how such principles can be applied to current models. In particular, we assess how concepts such as locality and globality apply to recent forecasting architectures. We show that accounting for these aspects can be more relevant for achieving accurate results than adopting specific sequence modeling layers and that simple, well-designed forecasting architectures can often match the state of the art. We discuss how overlooked implementation details in existing architectures (1) fundamentally change the class of the resulting forecasting method and (2) drastically affect the observed empirical results. Our results call for rethinking current faulty benchmarking practices and the need to focus on the foundational aspects of the forecasting problem when designing architectures. As a step in this direction, we propose an auxiliary forecasting model card, whose fields serve to characterize existing and new forecasting architectures based on key design choices.
Paper Structure (29 sections, 5 equations, 4 figures, 19 tables)

This paper contains 29 sections, 5 equations, 4 figures, 19 tables.

Table of Contents

  1. Introduction
  2. Related work and context
  3. Preliminaries
  4. Problem setting
  5. Forecasting groups of time series
  6. Baselines
  7. Reference architectures
  8. Benchmarks
  9. What Matters in Deep Learning for Time Series Forecasting?
  10. Design dimension 1: Model configuration
  11. Design dimension 2: Preprocessing and exogenous variables
  12. Design dimension 3: Temporal processing
  13. Design dimension 4: Spatial processing
  14. Discussion
  15. Conclusion
  16. ...and 14 more sections

Figures (4)

  • Figure 1: mse versus mean batch time during training on the Electricity dataset for a forecasting horizon of $96$. Circle size indicates memory consumption.
  • Figure 2: Block diagram of the reference architectures
  • Figure 3: MAE and MSE performance versus mean batch time during training for models not including spatial processing, for a batch size of 512. Circle size indicates memory consumption.
  • Figure 4: MAE and MSE performance versus mean batch time during training for models including spatial processing, for a batch size of 32. Circle size indicates memory consumption.