vLinear: A Powerful Linear Model for Multivariate Time Series Forecasting

TL;DR

vLinear tackles multivariate time series forecasting with a lightweight, linear-time approach. It introduces vecTrans, a rank-1 vector-based module that models cross-variate correlations with $O(N)$ complexity, and WFMLoss, a final-series-oriented flow-matching objective with path- and horizon-weighting. Together, these components deliver state-of-the-art accuracy across 22 benchmarks and offer up to $5\times$ inference speedups and reduced FLOPs/memory when integrated into Transformer forecasters. The work also demonstrates strong generalizability by improving existing forecasters as plug-ins and provides theoretical justifications for the WFMLoss design.

Abstract

In this paper, we present \textbf{vLinear}, an effective yet efficient \textbf{linear}-based multivariate time series forecaster featuring two components: the \textbf{v}ecTrans module and the WFMLoss objective. Many state-of-the-art forecasters rely on self-attention or its variants to capture multivariate correlations, typically incurring $\mathcal{O}(N^2)$ computational complexity with respect to the number of variates $N$. To address this, we propose vecTrans, a lightweight module that utilizes a learnable vector to model multivariate correlations, reducing the complexity to $\mathcal{O}(N)$. Notably, vecTrans can be seamlessly integrated into Transformer-based forecasters, delivering up to 5$\times$ inference speedups and consistent performance gains. Furthermore, we introduce WFMLoss (Weighted Flow Matching Loss) as the objective. In contrast to typical \textbf{velocity-oriented} flow matching objectives, we demonstrate that a \textbf{final-series-oriented} formulation yields significantly superior forecasting accuracy. WFMLoss also incorporates path- and horizon-weighted strategies to focus learning on more reliable paths and horizons. Empirically, vLinear achieves state-of-the-art performance across 22 benchmarks and 124 forecasting settings. Moreover, WFMLoss serves as an effective plug-and-play objective, consistently improving existing forecasters. The code is available at https://anonymous.4open.science/r/vLinear.

Figures (13)

• Figure 1: Comparison of multi-head self-attention (MHSA), NormLin olinear, and vecTrans. vecTrans employs a learnable rank-1 matrix to model token dependencies. By further re-arranging the computational order, vecTrans reduces the complexity from $\mathcal{O}(N^2)$ to $\mathcal{O}(N)$. When integrated into Transformer-based forecasters (e.g., iTransformer), vecTrans yields up to $5\times$ inference speedup while consistently improving forecasting accuracy.
• Figure 2: Illustration of the typical flow matching loss and WFMLoss. Unlike velocity-oriented alignment, WFMLoss focuses on calibrating the final predicted series and incorporates path- and horizon-weighted strategies to improve forecasting performance.
• Figure 3: Overall architecture of vLinear, featuring the vecTrans module and the WFMLoss design. vecTrans uses a single learnable vector to aggregate multivariate features and broadcasts the aggregated representation to each variate by expanding along the variate dimension. WFMLoss is based on final-series-oriented flow matching and incorporates path- and horizon-weighted strategies.
• Figure 4: Model efficiency comparison. Bubble areas represent GPU memory usage during inference. The 'Input-96-Predict-96' setting is used. Resource footprint data are from Table \ref{['tab_gpu']}.
• Figure 5: Visualization of our forecasting results.

Theorems & Definitions (9)

• Theorem 1: Rank-1 Row-Normalized Matrix
• Theorem 2: Horizon-Weighted MAE Loss
• Theorem 3: Simplified Inference
• Theorem 4: Rank-1 Row-Normalized Matrix
• proof
• Theorem 5: Horizon-Weighted MAE Loss
• proof
• Theorem 6: Simplified Inference
• proof