SEER: Transformer-based Robust Time Series Forecasting via Automated Patch Enhancement and Replacement

TL;DR

SEER addresses the challenge of robust time series forecasting under real-world data degradations by introducing a dual-module Transformer framework. The Augmented Embedding Module enriches patch representations via a Mixture-of-Experts and a channel-adaptive series token mechanism, while the Learnable Patch Replacement Module dynamically filters and substitutes low-quality patches through a causal-attention-based scheme. Together, these components yield state-of-the-art accuracy and enhanced robustness across multiple datasets and perturbations, including missing values, anomalies, distribution shifts, and white noise. The approach demonstrates strong practical impact for real-world forecasting in domains with noisy or evolving data, such as IoT, energy, and finance.

Abstract

Time series forecasting is important in many fields that require accurate predictions for decision-making. Patching techniques, commonly used and effective in time series modeling, help capture temporal dependencies by dividing the data into patches. However, existing patch-based methods fail to dynamically select patches and typically use all patches during the prediction process. In real-world time series, there are often low-quality issues during data collection, such as missing values, distribution shifts, anomalies and white noise, which may cause some patches to contain low-quality information, negatively impacting the prediction results. To address this issue, this study proposes a robust time series forecasting framework called SEER. Firstly, we propose an Augmented Embedding Module, which improves patch-wise representations using a Mixture-of-Experts (MoE) architecture and obtains series-wise token representations through a channel-adaptive perception mechanism. Secondly, we introduce a Learnable Patch Replacement Module, which enhances forecasting robustness and model accuracy through a two-stage process: 1) a dynamic filtering mechanism eliminates negative patch-wise tokens; 2) a replaced attention module substitutes the identified low-quality patches with global series-wise token, further refining their representations through a causal attention mechanism. Comprehensive experimental results demonstrate the SOTA performance of SEER.

Figures (6)

• Figure 1: Common Scenarios of Low-Quality Datasets.
• Figure 2: To assess SEER's robustness under low-quality data conditions---see Section \ref{['Low-Quality']}, we inject varying proportions of missing values, distribution shifts, anomalies, and white noise into the ETTh2. We evaluate SRSNet, Amplifier, iTransformer, and DLinear as baselines. SRSNet inherently handles non-stationarity, while the others are enhanced with RobustTSF cheng2024robusttsf for improved robustness.
• Figure 3: The overall architecture of SEER primarily comprises the Augmented Embedding Module and the Learnable Patch Replacement Module, the latter of which integrates Automated Token Filtering and Replaced Attention mechanisms.
• Figure 4: Univariate forecasting results.
• Figure 5: Parameter sensitivity studies of main hyper-parameters in SEER.

Theorems & Definitions (5)

• Definition 3.1: Time series
• Definition 3.2: White Noise
• Definition 3.3: Anomalies
• Definition 3.4: Missing Values
• Definition 3.5: Distribution Shift qiu2025duet