CPiRi: Channel Permutation-Invariant Relational Interaction for Multivariate Time Series Forecasting

TL;DR

CPiRi tackles channel permutation brittleness in multivariate time series forecasting by decoupling temporal and spatial learning and enforcing permutation invariance through channel shuffling. It combines a frozen univariate temporal encoder (Sundial) with a lightweight, permutation-equivariant spatial module, trained under a loss that averages over all channel permutations. This yields state-of-the-art accuracy on several benchmarks, strong inductive generalization to unseen channels, and robust performance under channel reordering, while remaining computationally efficient on large-scale data. The work provides theoretical grounding for permutation equivariance and demonstrates practical impact for deployments facing structural and distributional co-drift in real-world sensor networks.

Abstract

Current methods for multivariate time series forecasting can be classified into channel-dependent and channel-independent models. Channel-dependent models learn cross-channel features but often overfit the channel ordering, which hampers adaptation when channels are added or reordered. Channel-independent models treat each channel in isolation to increase flexibility, yet this neglects inter-channel dependencies and limits performance. To address these limitations, we propose \textbf{CPiRi}, a \textbf{channel permutation invariant (CPI)} framework that infers cross-channel structure from data rather than memorizing a fixed ordering, enabling deployment in settings with structural and distributional co-drift without retraining. CPiRi couples \textbf{spatio-temporal decoupling architecture} with \textbf{permutation-invariant regularization training strategy}: a frozen pretrained temporal encoder extracts high-quality temporal features, a lightweight spatial module learns content-driven inter-channel relations, while a channel shuffling strategy enforces CPI during training. We further \textbf{ground CPiRi in theory} by analyzing permutation equivariance in multivariate time series forecasting. Experiments on multiple benchmarks show state-of-the-art results. CPiRi remains stable when channel orders are shuffled and exhibits strong \textbf{inductive generalization} to unseen channels even when trained on \textbf{only half} of the channels, while maintaining \textbf{practical efficiency} on large-scale datasets. The source code is released at https://github.com/JasonStraka/CPiRi.

Figures (10)

• Figure 1: Conceptual overview of the CPiRi framework. Top: channel-dependent models often overfit to channel order, leading to performance collapse in channel permutation invariance (CPI) tests. Middle: channel-independent models lack cross-channel interaction. Bottom: CPiRi resolves this by (1) using a frozen encoder for robust temporal feature extraction, and (2) training a channel permutation-equivariant spatial module with a channel shuffling strategy to learn generalizable, content-based relationships, ensuring both high accuracy and near-invariant robustness under channel shuffling.
• Figure 2: Architectural overview of the CPiRi framework. CPiRi operates in three stages: (1) A frozen univariate foundation model (Sundial Encoder) independently extracts temporal features for each channel. (2) A lightweight, trainable spatial module processes the set of channel representations to model permutation-invariant interactions. (3) The frozen Sundial Decoder independently generates the final forecast for each channel from its updated representation.
• Figure 3: Channel-level permutation equivariance. Rotating or reflecting the same directed graph permutes node indices while preserving edges, yielding an equivalent adjacency matrix. This illustrates that MTSF is permutation-equivariant at the channel level, and that CPI models can adapt to dynamic multivariate time series settings with reordered, added, or removed channels.
• Figure 4: Analysis of inductive generalization to unseen channels and the impact of permutation-invariant regularization strategy. Performance is evaluated with and without the strategy as the percentage of available training channels varies. The results show that the strategy consistently improves performance, with the benefits becoming more pronounced in low-data regimes. For clarity, WAPE (%) values are annotated directly on the corresponding data points.
• Figure 5: UMAP visualization of channel representations on METR-LA. Left: Sundial. Right: CPiRi under three random channel permutations, near-identical geometry with clearer separation.