C3RL: Rethinking the Combination of Channel-independence and Channel-mixing from Representation Learning

TL;DR

This work tackles multivariate time series forecasting (MTSF) by integrating channel-mixing (CM) and channel-independence (CI) strategies. It introduces C3RL, a SimSiam-inspired representation-learning framework that treats CM and CI inputs as transposed views in a Siamese network, jointly optimizing a contrastive loss and a forecasting loss with adaptive weighting. C3RL is model-agnostic and yields consistent improvements across nine public datasets when integrated with five CI-based models and two CM-based models, including substantial gains up to 81.4% in best-case performance for CI models and 76.3% for CM models. The approach enhances representation quality and generalization, offering a practical paradigm for improving MTSF models through cross-view representation learning and adaptive loss balancing.

Abstract

Multivariate time series forecasting has drawn increasing attention due to its practical importance. Existing approaches typically adopt either channel-mixing (CM) or channel-independence (CI) strategies. CM strategy can capture inter-variable dependencies but fails to discern variable-specific temporal patterns. CI strategy improves this aspect but fails to fully exploit cross-variable dependencies like CM. Hybrid strategies based on feature fusion offer limited generalization and interpretability. To address these issues, we propose C3RL, a novel representation learning framework that jointly models both CM and CI strategies. Motivated by contrastive learning in computer vision, C3RL treats the inputs of the two strategies as transposed views and builds a siamese network architecture: one strategy serves as the backbone, while the other complements it. By jointly optimizing contrastive and prediction losses with adaptive weighting, C3RL balances representation and forecasting performance. Extensive experiments on seven models show that C3RL boosts the best-case performance rate to 81.4% for models based on CI strategy and to 76.3% for models based on CM strategy, demonstrating strong generalization and effectiveness.

Figures (16)

• Figure 1: The application of the simple siamese networks (SimSiam) from the images (left) to the time series (right).
• Figure 2: The pipeline of the combination of CI and CM by representation learning (C3RL). The T-Modules denotes the temporal modules
• Figure 3: The application of C3RL to the iTransformer.
• Figure 4: The structure of the Siamese Projection (left) and Prediction module (right).
• Figure 5: Visualization of the weights of DLinear on several datasets. Out-720 denotes that the prediction horizon is 720. Season means the seasonal item obtained by decomposition.