Deep Time Series Models: A Comprehensive Survey and Benchmark

TL;DR

This work comprehensively surveys deep time series models by separating foundational modules from architectural designs, highlighting how stationarization, decomposition, and Fourier-based representations underpin modern architectures. It then presents Time Series Library (TSLib), a fair benchmark implementing 30 models across 30 real-world datasets and four primary tasks to enable rigorous, cross-task comparisons. Through extensive experiments, the paper demonstrates that model choice should be task-aligned, with Transformer-based backbones excelling in forecasting while CNNs and MLPs offer strengths in other tasks and in efficiency. It closes with forward-looking directions on pre-training, foundation models, and the integration of exogenous data and heterogeneity, underscoring practical opportunities for industry-scale temporal modeling.

Abstract

Time series, characterized by a sequence of data points organized in a discrete-time order, are ubiquitous in real-world scenarios. Unlike other data modalities, time series present unique challenges due to their intricate and dynamic nature, including the entanglement of nonlinear patterns and time-variant trends. Analyzing such data is of great significance in practical applications and has been extensively studied for centuries. Recent years have witnessed remarkable breakthroughs in the time series community, with techniques shifting from traditional statistical methods to contemporary deep learning models. In this paper, we delve into the design of deep time series models across various analysis tasks and review the existing literature from two perspectives: basic modules and model architectures. Further, we develop and release Time Series Library (TSLib) as a fair benchmark of deep time series models for diverse analysis tasks. TSLib implements 30 prominent models, covers 30 datasets from different domains, and supports five prevalent analysis tasks. Based on TSLib, we thoroughly evaluate 13 advanced deep time series models across diverse tasks. Empirical results indicate that models with specific structures are well-suited for distinct analytical tasks, providing insights for research and adoption of deep time series models. Code and datasets are available at https://github.com/thuml/Time-Series-Library.

Figures (10)

• Figure 1: Schematic illustration of different time series analysis tasks.
• Figure 2: An overview of representative time series models in chronological order. We mark models with different colors based on their architectures.
• Figure 3: Illustration of a basic MLP-based model in forecasting task, which captures the future-past dependencies with learnable MLP parameters.
• Figure 4: Illustration of RNN-based model in the forecasting task.
• Figure 5: Comparison of existing CNN-based time series models from the perspective of representation space.