Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

XLinear: Frequency-Enhanced MLP with CrossFilter for Robust Long-Range Forecasting

Xiang Ao

Abstract

Time series forecasters are widely used across various domains. Among them, MLP (multi-layer perceptron)-based forecasters have been proven to be more robust to noise compared to Transformer-based forecasters. However, MLP struggles to capture complex features, resulting in limitations on capturing long-range dependencies. To address this challenge, we propose XLinear, an MLP-based forecaster for long-range forecasting. Firstly, we decompose the time series into trend and seasonal components. For the trend component which contains long-range characteristics, we design Enhanced Frequency Attention (EFA) to capture long-term dependencies by leveraging frequency-domain operations. Additionally, a CrossFilter Block is proposed for the seasonal component to maintain the model's robustness to noise, avoiding the problems of low robustness often caused by attention mechanisms. Experimental results demonstrate that XLinear achieves state-of-the-art performance on test datasets. While keeping the lightweight architecture and high robustness of MLP-based models, our forecaster outperforms other MLP-based forecasters in capturing long-range dependencies.

XLinear: Frequency-Enhanced MLP with CrossFilter for Robust Long-Range Forecasting

Abstract

Time series forecasters are widely used across various domains. Among them, MLP (multi-layer perceptron)-based forecasters have been proven to be more robust to noise compared to Transformer-based forecasters. However, MLP struggles to capture complex features, resulting in limitations on capturing long-range dependencies. To address this challenge, we propose XLinear, an MLP-based forecaster for long-range forecasting. Firstly, we decompose the time series into trend and seasonal components. For the trend component which contains long-range characteristics, we design Enhanced Frequency Attention (EFA) to capture long-term dependencies by leveraging frequency-domain operations. Additionally, a CrossFilter Block is proposed for the seasonal component to maintain the model's robustness to noise, avoiding the problems of low robustness often caused by attention mechanisms. Experimental results demonstrate that XLinear achieves state-of-the-art performance on test datasets. While keeping the lightweight architecture and high robustness of MLP-based models, our forecaster outperforms other MLP-based forecasters in capturing long-range dependencies.
Paper Structure (14 sections, 22 equations, 7 figures, 1 table)

This paper contains 14 sections, 22 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. PRELIMINARIES
  3. Time Series Decomposition
  4. Attention Mechanism
  5. Fast Fourier Transform
  6. METHODOLOGY
  7. Model Structure
  8. EFA(Enhanced Frequency Attention) Block
  9. CrossFilter Block
  10. EXPERIMENTS
  11. Experimental Setup
  12. Main Results
  13. Model Analysis
  14. CONCLUSION

Figures (7)

  • Figure 1: Framework of our XLinear. The 'Time series decomp' is moving average time series decomposition in TimesNet.17 The seasonal component is input to CrossFilter Block and trend component is input into EFA Block. The outputs of these two blocks are concatenated and then processed by an MLP layer.
  • Figure 2: EFA vs. other attention. (a) Other attention:every components of the series are strengthened. (b) EFA: only low-frequency component is strengthened.
  • Figure 3: Integration of PaiFilter and TexFilter. (a) Direct concatenation of two filters: Noise is superimposed when simply combining results; (b) Frequency-domain multiplication: Noise is filtered rather than aggregated.
  • Figure 4: Results of different sequence length on 3 datasets.
  • Figure 5: Visualization of filter results. 100 of steps are presented here. ASB refers to Adaptive Spectral Block.15 A portion of the results was extracted for demonstration.
  • ...and 2 more figures