Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FRWKV:Frequency-Domain Linear Attention for Long-Term Time Series Forecasting

Qingyuan Yang, Shizhuo Deng, Dongyue Chen, Da Teng, Zehua Gan

TL;DR

The paper tackles the inefficiency of quadratic self-attention in long-horizon time-series forecasting. It introduces FRWKV, a frequency-domain linear-attention framework that applies RWKV-style state recursion to real and imaginary components of rFFT spectra, then reconstructs forecasts via irFFT. Empirically, FRWKV achieves competitive or state-of-the-art results across eight real-world datasets, with particular gains at long horizons, and ablation studies confirm the value of both frequency-domain processing and linear attention. This work demonstrates a scalable, spectrally informed approach to time-series forecasting that bridges efficiency and accuracy.

Abstract

Traditional Transformers face a major bottleneck in long-sequence time series forecasting due to their quadratic complexity $(\mathcal{O}(T^2))$ and their limited ability to effectively exploit frequency-domain information. Inspired by RWKV's $\mathcal{O}(T)$ linear attention and frequency-domain modeling, we propose FRWKV, a frequency-domain linear-attention framework that overcomes these limitations. Our model integrates linear attention mechanisms with frequency-domain analysis, achieving $\mathcal{O}(T)$ computational complexity in the attention path while exploiting spectral information to enhance temporal feature representations for scalable long-sequence modeling. Across eight real-world datasets, FRWKV achieves a first-place average rank. Our ablation studies confirm the critical roles of both the linear attention and frequency-encoder components. This work demonstrates the powerful synergy between linear attention and frequency analysis, establishing a new paradigm for scalable time series modeling. Code is available at this repository: https://github.com/yangqingyuan-byte/FRWKV.

FRWKV:Frequency-Domain Linear Attention for Long-Term Time Series Forecasting

TL;DR

The paper tackles the inefficiency of quadratic self-attention in long-horizon time-series forecasting. It introduces FRWKV, a frequency-domain linear-attention framework that applies RWKV-style state recursion to real and imaginary components of rFFT spectra, then reconstructs forecasts via irFFT. Empirically, FRWKV achieves competitive or state-of-the-art results across eight real-world datasets, with particular gains at long horizons, and ablation studies confirm the value of both frequency-domain processing and linear attention. This work demonstrates a scalable, spectrally informed approach to time-series forecasting that bridges efficiency and accuracy.

Abstract

Traditional Transformers face a major bottleneck in long-sequence time series forecasting due to their quadratic complexity and their limited ability to effectively exploit frequency-domain information. Inspired by RWKV's linear attention and frequency-domain modeling, we propose FRWKV, a frequency-domain linear-attention framework that overcomes these limitations. Our model integrates linear attention mechanisms with frequency-domain analysis, achieving computational complexity in the attention path while exploiting spectral information to enhance temporal feature representations for scalable long-sequence modeling. Across eight real-world datasets, FRWKV achieves a first-place average rank. Our ablation studies confirm the critical roles of both the linear attention and frequency-encoder components. This work demonstrates the powerful synergy between linear attention and frequency analysis, establishing a new paradigm for scalable time series modeling. Code is available at this repository: https://github.com/yangqingyuan-byte/FRWKV.

Paper Structure

This paper contains 14 sections, 7 equations, 1 figure, 3 tables.

Table of Contents

  1. INTRODUCTION
  2. METHODOLOGY
  3. Problem Definition
  4. Model Overview
  5. Frequency-Domain Transformation
  6. Linear Attention in Frequency Encoders
  7. Token shift and mixing.
  8. State-recursive update.
  9. Bonus coupling and gating.
  10. EXPERIMENTS
  11. Experimental Setup
  12. Main Results
  13. Ablation Study
  14. CONCLUSION

Figures (1)

  • Figure 1: FRWKV architecture: The input time series undergoes normalization and embedding. The embedding is transformed to real and imaginary components by real-valued Fast Fourier Transform (rFFT). The real and imaginary components are processed separately through linear attention mechanisms, then concatenated and transformed back to the time domain via inverse real-valued Fast Fourier Transform (irFFT) and normalization to yield the final output.