PHAT: Modeling Period Heterogeneity for Multivariate Time Series Forecasting

TL;DR

This work addresses the challenge of period heterogeneity in multivariate time series forecasting by introducing PHAT, a Period Heterogeneity-Aware Transformer. PHAT detects dominant periods via FFT, organizes variates into period buckets, and folds sequences into a 3D periodic representation to learn bucket-specific dynamics. A novel Positive-Negative Attention (PNA) with an X-shaped receptive field models both phase-aligned and offset periodic dependencies, including a periodic modulation term that biases attention according to autocorrelation structure. Empirical results across 14 real-world datasets against 18 baselines demonstrate state-of-the-art performance on the majority of metrics, alongside substantial efficiency gains and interpretable visualizations of learned periodic components.

Abstract

While existing multivariate time series forecasting models have advanced significantly in modeling periodicity, they largely neglect the periodic heterogeneity common in real-world data, where variates exhibit distinct and dynamically changing periods. To effectively capture this periodic heterogeneity, we propose PHAT (Period Heterogeneity-Aware Transformer). Specifically, PHAT arranges multivariate inputs into a three-dimensional "periodic bucket" tensor, where the dimensions correspond to variate group characteristics with similar periodicity, time steps aligned by phase, and offsets within the period. By restricting interactions within buckets and masking cross-bucket connections, PHAT effectively avoids interference from inconsistent periods. We also propose a positive-negative attention mechanism, which captures periodic dependencies from two perspectives: periodic alignment and periodic deviation. Additionally, the periodic alignment attention scores are decomposed into positive and negative components, with a modulation term encoding periodic priors. This modulation constrains the attention mechanism to more faithfully reflect the underlying periodic trends. A mathematical explanation is provided to support this property. We evaluate PHAT comprehensively on 14 real-world datasets against 18 baselines, and the results show that it significantly outperforms existing methods, achieving highly competitive forecasting performance. Our sources is available at GitHub.

Figures (6)

• Figure 1: The visualization of period heterogeneity phenomenon on ZafNoo dataset. The orange region in autocorrelation function represents the 95% confidence interval for the null hypothesis from Bartlett's Test arsham2011bartlett. Correlation coefficients that fall within this interval are not statistically significant and cannot be rejected as noise chandler1987introduction.
• Figure 2: The overall architecture of PHAT. Each bucket contains the same periodic variates and uses PNA to capture the periodic attention mechanism.
• Figure 3: Ablation study and hyperparameters sensitivity experiments of phats
• Figure 4: Multi-level visualizations on raw-sequence level, feature level and attention level.
• Figure 5: Autocorrelation function (left) and periodic-offset attention weights (right). Highlighted regions denote positive components; dark regions denote negative components.