Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

InvDec: Inverted Decoder for Multivariate Time Series Forecasting with Separated Temporal and Variate Modeling

Yuhang Wang

TL;DR

InvDec introduces a principled separation between temporal encoding and variate-level decoding for multivariate time series forecasting by pairing a patch-based temporal encoder with an inverted decoder that performs variate-wise attention. Delayed variate embeddings and an adaptive residual fusion mechanism enable the model to enrich variable representations without distorting temporal features, yielding strong gains on high-dimensional datasets while remaining competitive on low-dimensional settings. Across seven benchmarks, InvDec-PatchTST achieves up to $20.9\%$ $MSE$ reduction on Electricity and notable improvements on Weather and Traffic, with ablations confirming the contribution of each component and an analysis showing benefits grow with dimensionality. These results highlight the importance of cross-variate modeling in high-dimensional contexts and demonstrate the practicality of a plug-and-play temporal–variate separation for scalable forecasting.

Abstract

Multivariate time series forecasting requires simultaneously modeling temporal patterns and cross-variate dependencies. Channel-independent methods such as PatchTST excel at temporal modeling but ignore variable correlations, while pure variate-attention approaches such as iTransformer sacrifice temporal encoding. We proposeInvDec (Inverted Decoder), a hybrid architecture that achieves principled separation between temporal encoding and variate-level decoding. InvDec combines a patch-based temporal encoder with an inverted decoder operating on the variate dimension through variate-wise self-attention. We introduce delayed variate embeddings that enrich variable-specific representations only after temporal encoding, preserving temporal feature integrity. An adaptive residual fusion mechanism dynamically balances temporal and variate information across datasets of varying dimensions. Instantiating InvDec with PatchTST yields InvDec-PatchTST. Extensive experiments on seven benchmarks demonstrate significant gains on high-dimensional datasets: 20.9% MSE reduction on Electricity (321 variables), 4.3% improvement on Weather, and 2.7% gain on Traffic compared to PatchTST, while maintaining competitive performance on low-dimensional ETT datasets. Ablation studies validate each component, and analysis reveals that InvDec's advantage grows with dataset dimensionality, confirming that cross-variate modeling becomes critical as the number of variables increases.

InvDec: Inverted Decoder for Multivariate Time Series Forecasting with Separated Temporal and Variate Modeling

TL;DR

InvDec introduces a principled separation between temporal encoding and variate-level decoding for multivariate time series forecasting by pairing a patch-based temporal encoder with an inverted decoder that performs variate-wise attention. Delayed variate embeddings and an adaptive residual fusion mechanism enable the model to enrich variable representations without distorting temporal features, yielding strong gains on high-dimensional datasets while remaining competitive on low-dimensional settings. Across seven benchmarks, InvDec-PatchTST achieves up to reduction on Electricity and notable improvements on Weather and Traffic, with ablations confirming the contribution of each component and an analysis showing benefits grow with dimensionality. These results highlight the importance of cross-variate modeling in high-dimensional contexts and demonstrate the practicality of a plug-and-play temporal–variate separation for scalable forecasting.

Abstract

Multivariate time series forecasting requires simultaneously modeling temporal patterns and cross-variate dependencies. Channel-independent methods such as PatchTST excel at temporal modeling but ignore variable correlations, while pure variate-attention approaches such as iTransformer sacrifice temporal encoding. We proposeInvDec (Inverted Decoder), a hybrid architecture that achieves principled separation between temporal encoding and variate-level decoding. InvDec combines a patch-based temporal encoder with an inverted decoder operating on the variate dimension through variate-wise self-attention. We introduce delayed variate embeddings that enrich variable-specific representations only after temporal encoding, preserving temporal feature integrity. An adaptive residual fusion mechanism dynamically balances temporal and variate information across datasets of varying dimensions. Instantiating InvDec with PatchTST yields InvDec-PatchTST. Extensive experiments on seven benchmarks demonstrate significant gains on high-dimensional datasets: 20.9% MSE reduction on Electricity (321 variables), 4.3% improvement on Weather, and 2.7% gain on Traffic compared to PatchTST, while maintaining competitive performance on low-dimensional ETT datasets. Ablation studies validate each component, and analysis reveals that InvDec's advantage grows with dataset dimensionality, confirming that cross-variate modeling becomes critical as the number of variables increases.
Paper Structure (18 sections, 18 equations, 3 figures, 5 tables)

This paper contains 18 sections, 18 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Deep Learning Architectures for Time Series Forecasting
  4. Transformer-based Time Series Forecasting
  5. Cross-Variate Modeling and Novel Forecasting Paradigms
  6. Methodology
  7. Problem Formulation
  8. InvDec: Inverted Decoder Architecture
  9. Comparison with Related Architectures
  10. Experiments
  11. Experimental Setup
  12. Main Results
  13. Ablation Study
  14. Model Analysis
  15. Computational Complexity Analysis
  16. ...and 3 more sections

Figures (3)

  • Figure 1: Performance overview: InvDec-PatchTST (red) consistently outperforms baselines on high-dimensional datasets (Electricity, Weather, Traffic) while remaining competitive on low-dimensional ETT benchmarks. Radial distance represents normalized MSE.
  • Figure 2: Architecture of InvDec-PatchTST. The temporal encoder (green) captures patch-wise temporal patterns in a channel-independent manner, while the inverted decoder (red) models cross-variate dependencies through variate-wise self-attention. Delayed variate embeddings $\mathbf{E}_{\textrm{var}}$ are introduced only after temporal encoding to preserve temporal feature integrity. An adaptive residual fusion mechanism (yellow, red dashed path) with learnable weight $\lambda$ dynamically balances contributions from both pathways.
  • Figure 3: Relative MSE improvement (%) of InvDec-PatchTST over PatchTST baseline across datasets with different dimensionalities. InvDec's advantage increases with the number of variables, validating its design for high-dimensional scenarios.