Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CMoS: Rethinking Time Series Prediction Through the Lens of Chunk-wise Spatial Correlations

Haotian Si, Changhua Pei, Jianhui Li, Dan Pei, Gaogang Xie

TL;DR

CMoS is presented, a super-lightweight time series forecasting model that directly models the spatial correlations between different time series chunks, and introduces a Correlation Mixing technique that enables the model to capture diverse spatial correlations with minimal parameters.

Abstract

Recent advances in lightweight time series forecasting models suggest the inherent simplicity of time series forecasting tasks. In this paper, we present CMoS, a super-lightweight time series forecasting model. Instead of learning the embedding of the shapes, CMoS directly models the spatial correlations between different time series chunks. Additionally, we introduce a Correlation Mixing technique that enables the model to capture diverse spatial correlations with minimal parameters, and an optional Periodicity Injection technique to ensure faster convergence. Despite utilizing as low as 1% of the lightweight model DLinear's parameters count, experimental results demonstrate that CMoS outperforms existing state-of-the-art models across multiple datasets. Furthermore, the learned weights of CMoS exhibit great interpretability, providing practitioners with valuable insights into temporal structures within specific application scenarios.

CMoS: Rethinking Time Series Prediction Through the Lens of Chunk-wise Spatial Correlations

TL;DR

CMoS is presented, a super-lightweight time series forecasting model that directly models the spatial correlations between different time series chunks, and introduces a Correlation Mixing technique that enables the model to capture diverse spatial correlations with minimal parameters.

Abstract

Recent advances in lightweight time series forecasting models suggest the inherent simplicity of time series forecasting tasks. In this paper, we present CMoS, a super-lightweight time series forecasting model. Instead of learning the embedding of the shapes, CMoS directly models the spatial correlations between different time series chunks. Additionally, we introduce a Correlation Mixing technique that enables the model to capture diverse spatial correlations with minimal parameters, and an optional Periodicity Injection technique to ensure faster convergence. Despite utilizing as low as 1% of the lightweight model DLinear's parameters count, experimental results demonstrate that CMoS outperforms existing state-of-the-art models across multiple datasets. Furthermore, the learned weights of CMoS exhibit great interpretability, providing practitioners with valuable insights into temporal structures within specific application scenarios.

Paper Structure

This paper contains 38 sections, 1 theorem, 15 equations, 13 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Development of lightweight time series forecasting models
  4. Channel Strategy for Lightweight Models
  5. CMoS
  6. Chunk-wise Spatial Correlation Modeling
  7. Correlation Mixing
  8. Periodicity Injection by Weight Editing
  9. Instance Normalization
  10. Loss Function
  11. Parameter Efficiency Analysis
  12. Experiments
  13. Setup
  14. Main Results
  15. Ablation Study
  16. ...and 23 more sections

Key Result

Theorem 3.2

Within each chunk, perform a weighted average of the point-to-point weights $\{\theta_1, \theta_2, \dots, \theta_{n}\}$ to obtain new weights $\theta^*=\frac{\sum_{i=1}^n{\alpha_i \theta_i}}{\sum_{i=1}^n{\alpha_i}} (\alpha_i \geq 0)$. Consequently, we have $\sigma^2\sum_{i=1}^n{\theta_i^2} \geq \sig

Figures (13)

  • Figure 1: As time advances, the specific patterns in the time window change greatly, while the spatial correlations of the time series chunks remain similar.
  • Figure 2: The spatial correlations of multiple time series can be represented by the combination of fewer sub-correlations.
  • Figure 3: CMoS Architecture.
  • Figure 4: Illustration of Periodicity Injection.
  • Figure 5: Test loss during training process on different datasets with horizon$=96$. PI is the abbreviation of Periodicity Injection. The model with Periodicity Injection exhibits faster loss reduction.
  • ...and 8 more figures

Theorems & Definitions (2)

  • Definition 3.1
  • Theorem 3.2