Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dualformer: Time-Frequency Dual Domain Learning for Long-term Time Series Forecasting

Jingjing Bai, Yoshinobu Kawahara

TL;DR

Dualformer is proposed, a principled dual-domain framework that rethinks frequency modeling from a layer-wise perspective and enables structured frequency modeling and adaptive integration of time-frequency features, effectively preserving high-frequency information and enhancing generalization.

Abstract

Transformer-based models, despite their promise for long-term time series forecasting (LTSF), suffer from an inherent low-pass filtering effect that limits their effectiveness. This issue arises due to undifferentiated propagation of frequency components across layers, causing a progressive attenuation of high-frequency information crucial for capturing fine-grained temporal variations. To address this limitation, we propose Dualformer, a principled dual-domain framework that rethinks frequency modeling from a layer-wise perspective. Dualformer introduces three key components: (1) a dual-branch architecture that concurrently models complementary temporal patterns in both time and frequency domains; (2) a hierarchical frequency sampling module that allocates distinct frequency bands to different layers, preserving high-frequency details in lower layers while modeling low-frequency trends in deeper layers; and (3) a periodicity-aware weighting mechanism that dynamically balances contributions from the dual branches based on the harmonic energy ratio of inputs, supported theoretically by a derived lower bound. This design enables structured frequency modeling and adaptive integration of time-frequency features, effectively preserving high-frequency information and enhancing generalization. Extensive experiments conducted on eight widely used benchmarks demonstrate Dualformer's robustness and superior performance, particularly on heterogeneous or weakly periodic data. Our code is publicly available at https://github.com/Akira-221/Dualformer.

Dualformer: Time-Frequency Dual Domain Learning for Long-term Time Series Forecasting

TL;DR

Dualformer is proposed, a principled dual-domain framework that rethinks frequency modeling from a layer-wise perspective and enables structured frequency modeling and adaptive integration of time-frequency features, effectively preserving high-frequency information and enhancing generalization.

Abstract

Transformer-based models, despite their promise for long-term time series forecasting (LTSF), suffer from an inherent low-pass filtering effect that limits their effectiveness. This issue arises due to undifferentiated propagation of frequency components across layers, causing a progressive attenuation of high-frequency information crucial for capturing fine-grained temporal variations. To address this limitation, we propose Dualformer, a principled dual-domain framework that rethinks frequency modeling from a layer-wise perspective. Dualformer introduces three key components: (1) a dual-branch architecture that concurrently models complementary temporal patterns in both time and frequency domains; (2) a hierarchical frequency sampling module that allocates distinct frequency bands to different layers, preserving high-frequency details in lower layers while modeling low-frequency trends in deeper layers; and (3) a periodicity-aware weighting mechanism that dynamically balances contributions from the dual branches based on the harmonic energy ratio of inputs, supported theoretically by a derived lower bound. This design enables structured frequency modeling and adaptive integration of time-frequency features, effectively preserving high-frequency information and enhancing generalization. Extensive experiments conducted on eight widely used benchmarks demonstrate Dualformer's robustness and superior performance, particularly on heterogeneous or weakly periodic data. Our code is publicly available at https://github.com/Akira-221/Dualformer.
Paper Structure (15 sections, 1 theorem, 32 equations, 11 figures, 8 tables, 3 algorithms)

This paper contains 15 sections, 1 theorem, 32 equations, 11 figures, 8 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Model
  4. Hierarchical frequency sampling
  5. Time branch
  6. Frequency branch
  7. Periodicity-aware weighting
  8. Experiments
  9. Datasets
  10. Baselines and experimental setup
  11. Main Results
  12. Ablation Study
  13. Hierarchical frequency sampling evaluation
  14. Periodicity-aware weighting evaluation
  15. Conclusion

Key Result

Theorem 1

Let $f(t)$ be a continuous time series defined on $[0, L]$, with its discrete form sampled as $[f(0),\cdots,f(L-1)]$. Assume that $f(t)$ can be decomposed into$f(t) = f_p(t) + f_r(t),$ where $f_p(t)$ is the strictly periodic component with period $\tau$, satisfying $L=m\tau, m\in \mathbb{N}^\ast$, i where $E_p$ is energy of periodic part, $E_r$ is energy of residual part, $E_f$ is total spectral e

Figures (11)

  • Figure 1: The overall architecture of our proposed Dualformer model.
  • Figure 2: Hierarchical frequency sampling. When $\alpha\le 1/N$, case 1 is adopted; otherwise, case 2 is adopted.
  • Figure 3: Ablation study results on ETTh1 dataset. Average results of all prediction lengths $T\in\{96,192,336,720\}$ on multivariate forecasting.
  • Figure 4: MSE across different frequency selection strategies in two heterogeneous datasets: weather and traffic.
  • Figure 5: Weight distribution for various time series across four distinct datasets. The examples represent different segments from the ETTh1 and Weather series, and different variables from the Electricity and Traffic datasets.
  • ...and 6 more figures

Theorems & Definitions (2)

  • Theorem 1
  • proof