Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Not All Frequencies Are Created Equal:Towards a Dynamic Fusion of Frequencies in Time-Series Forecasting

Xingyu Zhang, Siyu Zhao, Zeen Song, Huijie Guo, Jianqi Zhang, Changwen Zheng, Wenwen Qiang

TL;DR

Frequency Dynamic Fusion (FreDF), which individually predicts each Fourier component, and dynamically fuses the output of different frequencies, is designed and it is proved FreDF has a lower bound, indicating that FreDF has better generalization ability.

Abstract

Long-term time series forecasting is a long-standing challenge in various applications. A central issue in time series forecasting is that methods should expressively capture long-term dependency. Furthermore, time series forecasting methods should be flexible when applied to different scenarios. Although Fourier analysis offers an alternative to effectively capture reusable and periodic patterns to achieve long-term forecasting in different scenarios, existing methods often assume high-frequency components represent noise and should be discarded in time series forecasting. However, we conduct a series of motivation experiments and discover that the role of certain frequencies varies depending on the scenarios. In some scenarios, removing high-frequency components from the original time series can improve the forecasting performance, while in others scenarios, removing them is harmful to forecasting performance. Therefore, it is necessary to treat the frequencies differently according to specific scenarios. To achieve this, we first reformulate the time series forecasting problem as learning a transfer function of each frequency in the Fourier domain. Further, we design Frequency Dynamic Fusion (FreDF), which individually predicts each Fourier component, and dynamically fuses the output of different frequencies. Moreover, we provide a novel insight into the generalization ability of time series forecasting and propose the generalization bound of time series forecasting. Then we prove FreDF has a lower bound, indicating that FreDF has better generalization ability. Extensive experiments conducted on multiple benchmark datasets and ablation studies demonstrate the effectiveness of FreDF. The code is available at https://github.com/Zh-XY22/FreDF.

Not All Frequencies Are Created Equal:Towards a Dynamic Fusion of Frequencies in Time-Series Forecasting

TL;DR

Frequency Dynamic Fusion (FreDF), which individually predicts each Fourier component, and dynamically fuses the output of different frequencies, is designed and it is proved FreDF has a lower bound, indicating that FreDF has better generalization ability.

Abstract

Long-term time series forecasting is a long-standing challenge in various applications. A central issue in time series forecasting is that methods should expressively capture long-term dependency. Furthermore, time series forecasting methods should be flexible when applied to different scenarios. Although Fourier analysis offers an alternative to effectively capture reusable and periodic patterns to achieve long-term forecasting in different scenarios, existing methods often assume high-frequency components represent noise and should be discarded in time series forecasting. However, we conduct a series of motivation experiments and discover that the role of certain frequencies varies depending on the scenarios. In some scenarios, removing high-frequency components from the original time series can improve the forecasting performance, while in others scenarios, removing them is harmful to forecasting performance. Therefore, it is necessary to treat the frequencies differently according to specific scenarios. To achieve this, we first reformulate the time series forecasting problem as learning a transfer function of each frequency in the Fourier domain. Further, we design Frequency Dynamic Fusion (FreDF), which individually predicts each Fourier component, and dynamically fuses the output of different frequencies. Moreover, we provide a novel insight into the generalization ability of time series forecasting and propose the generalization bound of time series forecasting. Then we prove FreDF has a lower bound, indicating that FreDF has better generalization ability. Extensive experiments conducted on multiple benchmark datasets and ablation studies demonstrate the effectiveness of FreDF. The code is available at https://github.com/Zh-XY22/FreDF.
Paper Structure (21 sections, 3 theorems, 22 equations, 2 figures, 4 tables, 1 algorithm)

This paper contains 21 sections, 3 theorems, 22 equations, 2 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Empirical Analysis
  4. Experimental Setup
  5. Experimental Observations and Analysis
  6. Method
  7. Time series forecasting in Fourier domain
  8. Frequency Dynamic Fusion
  9. Embedding
  10. FDBlocks
  11. Projection
  12. Theoretical Analysis
  13. Experiments
  14. Main Results
  15. Baselines
  16. ...and 6 more sections

Key Result

theorem 1

(The convolution theorem katznelson2004introduction). The convolution theorem states that the Fourier transform of a convolution of two functions equals the point-wise product of their Fourier transform:

Figures (2)

  • Figure 1: The comparison of prediction results using different frequencies on different datasets. (a) ETTm1; (b)ETTm2; (c)ETTh1; (d)ETTh2; (e) Exchange-rate; (f) Weather. Eliminating certain frequencies may improve performance in specific datasets while decreasing in others.
  • Figure 2: Overall structure of FreDF, which consists of an Embedding module $f$ for embedding the feature dimension $C$ to $D$, a Projection module $g$ for projection back to $C$, and $L$ FDBlocks. In FDBlock, we decompose and forecast in the frequency domain, and dynamically fuse the prediction results for each Fourier component.

Theorems & Definitions (3)

  • theorem 1
  • theorem 2
  • theorem 3