EDformer: Embedded Decomposition Transformer for Interpretable Multivariate Time Series Predictions
Sanjay Chakraborty, Ibrahim Delibasoglu, Fredrik Heintz
TL;DR
The paper presents EDformer, an encoder-only Transformer for multivariate time-series forecasting that embeds a seasonal-trend decomposition and a reverse-embedding strategy to improve interpretability and efficiency. By decomposing inputs with a moving-average block, reconstructing seasonal signals across reverse dimensions, and applying multivariate self-attention with per-variable FFN, the model forecasts via $\,\hat{\mathbf{X}}_{t+h} = f(\mathbf{X}_{t-l})\$ and adds back the trend component. Empirical results on diverse benchmarks show state-of-the-art accuracy and reduced runtimes compared to leading transformers, with ablations confirming the benefits of decomposition and reverse embedding. Extensive explainability analyses (FA, FO, IG, SHAP, GS, WinIT) demonstrate robust, interpretable feature attributions, reinforcing trust in predictions for complex, high-dimensional time-series data.
Abstract
Time series forecasting is a crucial challenge with significant applications in areas such as weather prediction, stock market analysis, and scientific simulations. This paper introduces an embedded decomposed transformer, 'EDformer', for multivariate time series forecasting tasks. Without altering the fundamental elements, we reuse the Transformer architecture and consider the capable functions of its constituent parts in this work. Edformer first decomposes the input multivariate signal into seasonal and trend components. Next, the prominent multivariate seasonal component is reconstructed across the reverse dimensions, followed by applying the attention mechanism and feed-forward network in the encoder stage. In particular, the feed-forward network is used for each variable frame to learn nonlinear representations, while the attention mechanism uses the time points of individual seasonal series embedded within variate frames to capture multivariate correlations. Therefore, the trend signal is added with projection and performs the final forecasting. The EDformer model obtains state-of-the-art predicting results in terms of accuracy and efficiency on complex real-world time series datasets. This paper also addresses model explainability techniques to provide insights into how the model makes its predictions and why specific features or time steps are important, enhancing the interpretability and trustworthiness of the forecasting results.