xPatch: Dual-Stream Time Series Forecasting with Exponential Seasonal-Trend Decomposition

TL;DR

This work tackles long-term time-series forecasting by addressing limitations of transformer-based models in capturing temporal structure. It introduces xPatch, a dual-stream CNN+MLP model that applies Exponential Moving Average (EMA) based seasonal-trend decomposition and patching with channel-independence to process multivariate data. A robust arctangent loss and a novel sigmoid learning rate schedule are proposed to improve training stability and performance. Empirical results on nine real-world datasets show that xPatch outperforms strong baselines (e.g., CARD, TimeMixer, PatchTST) under unified settings and hyperparameter search, offering a non-transformer, efficient alternative for LTSF with strong predictive accuracy.

Abstract

In recent years, the application of transformer-based models in time-series forecasting has received significant attention. While often demonstrating promising results, the transformer architecture encounters challenges in fully exploiting the temporal relations within time series data due to its attention mechanism. In this work, we design eXponential Patch (xPatch for short), a novel dual-stream architecture that utilizes exponential decomposition. Inspired by the classical exponential smoothing approaches, xPatch introduces the innovative seasonal-trend exponential decomposition module. Additionally, we propose a dual-flow architecture that consists of an MLP-based linear stream and a CNN-based non-linear stream. This model investigates the benefits of employing patching and channel-independence techniques within a non-transformer model. Finally, we develop a robust arctangent loss function and a sigmoid learning rate adjustment scheme, which prevent overfitting and boost forecasting performance. The code is available at the following repository: https://github.com/stitsyuk/xPatch.

Figures (22)

• Figure 1: Example of SMA decomposition with kernel k = 25 on a 96-length sample from the ETTh1 dataset.
• Figure 2: Example of EMA decomposition with $\alpha = \{ 0.1, 0.3, 0.5, 0.7, 0.9, 1 \}$ on a 96-length sample from the ETTh1 dataset.
• Figure 3: xPatch Model Overview. Every univariate series is passed through exponential decomposition. Consequently, the trend and seasonal components are processed through the dual flow network.
• Figure 4: SMA and EMA smoothing and decomposition on a 96-length sample from the Traffic dataset.
• Figure 5: Forecasting performance (MAE), lookback window $L = 96$, prediction horizon $T = 96$, alpha $\alpha = \{ 0.1, 0.3, 0.5, 0.7, 0.9 \}$.