Adapformer: Adaptive Channel Management for Multivariate Time Series Forecasting

TL;DR

Adapformer tackles the challenging balance between channel-independent and channel-dependent learning in multivariate time series forecasting by introducing a dual-stage Transformer augmented with Adaptive Channel Enhancer (ACE), Adaptive Channel Forecaster (ACF), and SimBlock. ACE enriches per-channel embeddings with a low-rank temporal capacity, while ACF selectively uses the most relevant covariates for each target, guided by SimBlock's learned inter-variable correlations. Empirically, Adapformer achieves state-of-the-art results across seven real-world datasets, with strong improvements in both accuracy and noise robustness, and it remains scalable to high-dimensional data due to lightweight ACE/ACF components and a plug-and-play design. The work suggests promising directions, including efficiency optimizations, enhanced similarity measures, and potential integration with graph-based methods to further exploit inter-variable structures in forecasting tasks.

Abstract

In multivariate time series forecasting (MTSF), accurately modeling the intricate dependencies among multiple variables remains a significant challenge due to the inherent limitations of traditional approaches. Most existing models adopt either \textbf{channel-independent} (CI) or \textbf{channel-dependent} (CD) strategies, each presenting distinct drawbacks. CI methods fail to leverage the potential insights from inter-channel interactions, resulting in models that may not fully exploit the underlying statistical dependencies present in the data. Conversely, CD approaches often incorporate too much extraneous information, risking model overfitting and predictive inefficiency. To address these issues, we introduce the Adaptive Forecasting Transformer (\textbf{Adapformer}), an advanced Transformer-based framework that merges the benefits of CI and CD methodologies through effective channel management. The core of Adapformer lies in its dual-stage encoder-decoder architecture, which includes the \textbf{A}daptive \textbf{C}hannel \textbf{E}nhancer (\textbf{ACE}) for enriching embedding processes and the \textbf{A}daptive \textbf{C}hannel \textbf{F}orecaster (\textbf{ACF}) for refining the predictions. ACE enhances token representations by selectively incorporating essential dependencies, while ACF streamlines the decoding process by focusing on the most relevant covariates, substantially reducing noise and redundancy. Our rigorous testing on diverse datasets shows that Adapformer achieves superior performance over existing models, enhancing both predictive accuracy and computational efficiency, thus making it state-of-the-art in MTSF.

Figures (24)

• Figure 1: Channel Independent Strategy with more Robustness (Left) and Channel Dependent Strategy with more Model Capacity (Right). An appropriate channel management should balance in the middle.
• Figure 2: An overview of the proposed Adapformer architecture. The raw inputs are first embedded and subsequently refined by the Adaptive Channel Enhancer (ACE), which enriches each token’s representational capacity. Canonical Transformer encoders then captures dependencies among these enhanced tokens, producing encoded representations that are passed to the decoder - Adaptive Channel Forecaster (ACF) - to predict future time-series outputs. Meanwhile, a separate similarity block (SimBlock) processes the raw inputs to explicitly model inter-sequence relationships for future predictions, providing an auxiliary output used both for additional training supervision and to further guide the ACF’s forecasting process. the ACE module.
• Figure 3: Illustration of the Adaptive Channel Forecaster (ACF). To predict the $i$-th target variable, the model first selects $k-1$ covariates most correlated with the target among future variables according to the results from SimBlock, forming a set of $k$ inputs (including the $i$-th variable itself). These inputs are then processed through a simple MLP with skip connection to produce future sequences, from which only the $i$-th target variable is retained as the final output.
• Figure 4: Comparison of Mean Square Errors (MSE) over varying prediction lengths on three benchmark datasets (ETTh1, Solar, and PEMS-03). We illustrate the performance of Channel Independent (light blue), Adapformer (dark blue), and Channel Dependent (medium blue). Adapformer consistently outperforms the two native channel strategies across all tested scenarios.
• Figure 5: Forecasting performance with lookback lengths of $T=\{48,96,192,336,720\}$ with a fixed prediction horizon of 96 time steps. Results are compared among four Transformer-based models: Adapformer, iTransformer liu2023itransformer, PatchTST nie2022time, and CARD wang2024card, across two datasets: ETTh1 and Electricity (ECL).