Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AWGformer: Adaptive Wavelet-Guided Transformer for Multi-Resolution Time Series Forecasting

Wei Li

TL;DR

Given input $X ∈ R^{T×D}$ and target $Y ∈ R^{H×D}$, AWGformer integrates Adaptive Wavelet Decomposition, Cross-Scale Feature Fusion, Frequency-Aware Multi-Head Attention, and a Hierarchical Prediction Network to deliver accurate, multi-resolution forecasts. The method learns wavelet bases and decomposition levels, models cross-band interactions with learnable couplings, and allocates frequency-specific attention heads to capture scale-specific dynamics. The paper provides convergence-related guarantees and frames the learned wavelet-guided attention within classical signal-processing principles. Empirical results on ETT, Traffic, and Electricity benchmarks show state-of-the-art performance across horizons, along with robustness to missing data and detailed analyses of learned wavelets.

Abstract

Time series forecasting requires capturing patterns across multiple temporal scales while maintaining computational efficiency. This paper introduces AWGformer, a novel architecture that integrates adaptive wavelet decomposition with cross-scale attention mechanisms for enhanced multi-variate time series prediction. Our approach comprises: (1) an Adaptive Wavelet Decomposition Module (AWDM) that dynamically selects optimal wavelet bases and decomposition levels based on signal characteristics; (2) a Cross-Scale Feature Fusion (CSFF) mechanism that captures interactions between different frequency bands through learnable coupling matrices; (3) a Frequency-Aware Multi-Head Attention (FAMA) module that weights attention heads according to their frequency selectivity; (4) a Hierarchical Prediction Network (HPN) that generates forecasts at multiple resolutions before reconstruction. Extensive experiments on benchmark datasets demonstrate that AWGformer achieves significant average improvements over state-of-the-art methods, with particular effectiveness on multi-scale and non-stationary time series. Theoretical analysis provides convergence guarantees and establishes the connection between our wavelet-guided attention and classical signal processing principles.

AWGformer: Adaptive Wavelet-Guided Transformer for Multi-Resolution Time Series Forecasting

TL;DR

Given input and target , AWGformer integrates Adaptive Wavelet Decomposition, Cross-Scale Feature Fusion, Frequency-Aware Multi-Head Attention, and a Hierarchical Prediction Network to deliver accurate, multi-resolution forecasts. The method learns wavelet bases and decomposition levels, models cross-band interactions with learnable couplings, and allocates frequency-specific attention heads to capture scale-specific dynamics. The paper provides convergence-related guarantees and frames the learned wavelet-guided attention within classical signal-processing principles. Empirical results on ETT, Traffic, and Electricity benchmarks show state-of-the-art performance across horizons, along with robustness to missing data and detailed analyses of learned wavelets.

Abstract

Time series forecasting requires capturing patterns across multiple temporal scales while maintaining computational efficiency. This paper introduces AWGformer, a novel architecture that integrates adaptive wavelet decomposition with cross-scale attention mechanisms for enhanced multi-variate time series prediction. Our approach comprises: (1) an Adaptive Wavelet Decomposition Module (AWDM) that dynamically selects optimal wavelet bases and decomposition levels based on signal characteristics; (2) a Cross-Scale Feature Fusion (CSFF) mechanism that captures interactions between different frequency bands through learnable coupling matrices; (3) a Frequency-Aware Multi-Head Attention (FAMA) module that weights attention heads according to their frequency selectivity; (4) a Hierarchical Prediction Network (HPN) that generates forecasts at multiple resolutions before reconstruction. Extensive experiments on benchmark datasets demonstrate that AWGformer achieves significant average improvements over state-of-the-art methods, with particular effectiveness on multi-scale and non-stationary time series. Theoretical analysis provides convergence guarantees and establishes the connection between our wavelet-guided attention and classical signal processing principles.
Paper Structure (22 sections, 2 theorems, 3 equations, 4 figures, 2 tables)

This paper contains 22 sections, 2 theorems, 3 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Problem Formulation
  5. Adaptive Wavelet Decomposition Module (AWDM)
  6. Learnable Wavelet Transform
  7. Adaptive Level Selection
  8. Cross-Scale Feature Fusion (CSFF)
  9. Frequency-Aware Multi-Head Attention (FAMA)
  10. Frequency-Selective Heads
  11. Weighted Attention
  12. Hierarchical Prediction Network (HPN)
  13. Training Objective
  14. Theoretical Properties
  15. Experiments
  16. ...and 7 more sections

Key Result

Theorem 1

For any $f \in L^2(\mathbb{R})$ with bounded variation, the adaptive wavelet decomposition with $J$ levels achieves: $\|f - f_J\|_2 \leq C \cdot 2^{-J\alpha} \|f\|_{BV}$ where $\alpha$ depends on the smoothness of learned wavelets and $C$ is a constant.

Figures (4)

  • Figure 1: The AWGformer architecture. The model integrates (1) AWDM for adaptive decomposition, (2) Stacked encoder layers with CSFF and FAMA for cross-scale modeling, and (3) HPN for hierarchical forecasting. Visualizations show learned wavelets and frequency-aware attention maps.
  • Figure 2: Left: Adaptive Decomposition separating components. Right: The learned attention map $\mathbf{M}_h$.
  • Figure 3: Comparison of wavelet bases. Left: The learned wavelet (blue) adapts its envelope to the signal. Right: In the frequency domain, the learned basis exhibits a more concentrated energy distribution (shaded area), providing better spectral localization than the fixed Db4 basis.
  • Figure 4: Robustness analysis on ETTm1 with 30% missing data. AWGformer (blue) maintains superior stability across all horizons compared to baselines.

Theorems & Definitions (2)

  • Theorem 1: Approximation Guarantee
  • Lemma 1: Frequency Localization