FreDF: Learning to Forecast in the Frequency Domain

TL;DR

FreDF addresses the bias in the Direct Forecast objective arising from label autocorrelation by learning forecasts in the frequency domain. It aligns forecast and label sequences via a differentiable FFT-based loss, while preserving the Direct Forecast paradigm and maintaining model-agnostic applicability. The approach is supported by theory showing decorrelation of frequency components as $T\to\infty$ and by extensive experiments across datasets and backbones that demonstrate consistent performance gains. FreDF offers a practical, plug-in improvement for time-series forecasting, enhancing accuracy and robustness without sacrificing sample efficiency. The work also points to future directions in adopting alternative orthogonal bases and extending the method to broader signal-processing contexts.

Abstract

Time series modeling presents unique challenges due to autocorrelation in both historical data and future sequences. While current research predominantly addresses autocorrelation within historical data, the correlations among future labels are often overlooked. Specifically, modern forecasting models primarily adhere to the Direct Forecast (DF) paradigm, generating multi-step forecasts independently and disregarding label autocorrelation over time. In this work, we demonstrate that the learning objective of DF is biased in the presence of label autocorrelation. To address this issue, we propose the Frequency-enhanced Direct Forecast (FreDF), which mitigates label autocorrelation by learning to forecast in the frequency domain, thereby reducing estimation bias. Our experiments show that FreDF significantly outperforms existing state-of-the-art methods and is compatible with a variety of forecast models. Code is available at https://github.com/Master-PLC/FreDF.

Figures (20)

• Figure 1: Visualizing label autocorrelation in time series forecasting. (a) shows the generation process of time series with dependencies depicted as arrows. (b) shows the label correlation in the time domain, where each element $\rho_{i,j}$ indicates the partial correlation between $Y_i$ and $Y_j$ given $L$. (c-d) shows the label correlation in the frequency domain, where each element $\rho_{i,j}$ indicates the partial correlation between $F_i$ and $F_j$ given $L$, shown with the real (c) and imaginary part (d). Due to the symmetry inherent in FFT, the forecast length in the frequency domain is halved.
• Figure 1: Running time in the forward pass (left panel) and backward pass (right panel), shown with dashed lines for the average and shaded areas for 99.9% confidence intervals.
• Figure 2: The workflow of FreDF. Key operations in the time and frequency domains are highlighted in red and blue, respectively.
• Figure 3: Visualization of forecast sequence generated with and without FreDF in the time (a-b) and frequency (c-d) domains, using the iTransformer as the backbone model.
• Figure 4: Benefit of incorporating FreDF in varying models, shown with colored bars for means over forecast lengths (96, 192, 336, 720) and error bars for 99.9% confidence intervals.

Theorems & Definitions (11)

• Theorem 3.1: Bias of DF
• Definition 3.2: Discrete Fourier Transform, DFT
• Theorem 3.3: Decorrelation between frequency components
• Theorem B.1: Bias of vanilla DF, simplified
• proof
• Theorem B.2: Bias of vanillia DF
• proof
• Corollary B.3: Bias of vanilla DF, multivariate
• proof
• Theorem B.4: Decorrelation between frequency components