Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tailored Architectures for Time Series Forecasting: Evaluating Deep Learning Models on Gaussian Process-Generated Data

Victoria Hankemeier, Malte Schilling

TL;DR

The paper tackles the challenge of linking time-series characteristics to neural forecasting architectures by introducing Gaussian-Process–generated benchmarks (GP-TimeSet) with controllable patterns. It proposes TimeFlex, a modular model that processes trend in the time domain and seasonality via Fourier transforms, complemented by dilated convolutions and learnable fusion. Across extensive ablations and benchmarks, the work shows that separating seasonal and trend components and applying FFT to the seasonal part yields robust gains on non-stationary and highly periodic data, while simpler decomposed models remain competitive on smoother data. The findings advance practical guidance for architectural design and benchmarking, highlighting when frequency-domain processing and component-wise modeling yield benefits and outlining avenues for evaluating larger, more diverse datasets.

Abstract

Developments in Deep Learning have significantly improved time series forecasting by enabling more accurate modeling of complex temporal dependencies inherent in sequential data. The effectiveness of such models is often demonstrated on limited sets of specific real-world data. Although this allows for comparative analysis, it still does not demonstrate how specific data characteristics align with the architectural strengths of individual models. Our research aims at uncovering clear connections between time series characteristics and particular models. We introduce a novel dataset generated using Gaussian Processes, specifically designed to display distinct, known characteristics for targeted evaluations of model adaptability to them. Furthermore, we present TimeFlex, a new model that incorporates a modular architecture tailored to handle diverse temporal dynamics, including trends and periodic patterns. This model is compared to current state-of-the-art models, offering a deeper understanding of how models perform under varied time series conditions.

Tailored Architectures for Time Series Forecasting: Evaluating Deep Learning Models on Gaussian Process-Generated Data

TL;DR

The paper tackles the challenge of linking time-series characteristics to neural forecasting architectures by introducing Gaussian-Process–generated benchmarks (GP-TimeSet) with controllable patterns. It proposes TimeFlex, a modular model that processes trend in the time domain and seasonality via Fourier transforms, complemented by dilated convolutions and learnable fusion. Across extensive ablations and benchmarks, the work shows that separating seasonal and trend components and applying FFT to the seasonal part yields robust gains on non-stationary and highly periodic data, while simpler decomposed models remain competitive on smoother data. The findings advance practical guidance for architectural design and benchmarking, highlighting when frequency-domain processing and component-wise modeling yield benefits and outlining avenues for evaluating larger, more diverse datasets.

Abstract

Developments in Deep Learning have significantly improved time series forecasting by enabling more accurate modeling of complex temporal dependencies inherent in sequential data. The effectiveness of such models is often demonstrated on limited sets of specific real-world data. Although this allows for comparative analysis, it still does not demonstrate how specific data characteristics align with the architectural strengths of individual models. Our research aims at uncovering clear connections between time series characteristics and particular models. We introduce a novel dataset generated using Gaussian Processes, specifically designed to display distinct, known characteristics for targeted evaluations of model adaptability to them. Furthermore, we present TimeFlex, a new model that incorporates a modular architecture tailored to handle diverse temporal dynamics, including trends and periodic patterns. This model is compared to current state-of-the-art models, offering a deeper understanding of how models perform under varied time series conditions.

Paper Structure

This paper contains 31 sections, 15 equations, 4 figures, 10 tables.

Table of Contents

  1. Introduction
  2. State-of-the-Art and Related work
  3. SotA in Deep Learning for Time Series Forecasting
  4. MLP
  5. RNN
  6. CNN
  7. Transformer
  8. Transformer
  9. SSM
  10. Models using Prior Knowledge
  11. Mixed-Model Architectures
  12. Related Work on Surveys and Benchmarking Frameworks
  13. Fundamentals of time series data
  14. Dataset Generation using Gaussian Processes
  15. Gaussian Processes
  16. ...and 16 more sections

Figures (4)

  • Figure 1: Architecture of TimeFlex model. The red boxes are optional parts.
  • Figure 2: MAE of TimeFlex with/without RevIn, 1D- and 2D FFT.
  • Figure 3: Comparison of TimeFlex with TimeFlex_jpp, i.e. post-processing applied after the weighted sum.
  • Figure 4: Average runtime over number of parameters.

Theorems & Definitions (4)

  • Definition 1: Stochastic Process
  • Definition 2: Weak Stationarity
  • Definition 3: Time series
  • Definition 4