Channel-aware Contrastive Conditional Diffusion for Multivariate Probabilistic Time Series Forecasting

TL;DR

A generic channel-aware Contrastive Conditional Diffusion model entitled CCDM is proposed to achieve desirable Multivariate probabilistic forecasting, obviating the need for curated temporal conditioning inductive biases and offering theoretic insights on the benefits of such auxiliary contrastive training refinement.

Abstract

Forecasting faithful trajectories of multivariate time series from practical scopes is essential for reasonable decision-making. Recent methods majorly tailor generative conditional diffusion models to estimate the target temporal predictive distribution. However, it remains an obstacle to enhance the exploitation efficiency of given implicit temporal predictive information to bolster conditional diffusion learning. To this end, we propose a generic channel-aware Contrastive Conditional Diffusion model entitled CCDM to achieve desirable Multivariate probabilistic forecasting, obviating the need for curated temporal conditioning inductive biases. In detail, we first design a channel-centric conditional denoising network to manage intra-variate variations and cross-variate correlations, which can lead to scalability on diverse prediction horizons and channel numbers. Then, we devise an ad-hoc denoising-based temporal contrastive learning to explicitly amplify the predictive mutual information between past observations and future forecasts. It can coherently complement naive step-wise denoising diffusion training and improve the forecasting accuracy and generality on unknown test time series. Besides, we offer theoretic insights on the benefits of such auxiliary contrastive training refinement from both neural mutual information and temporal distribution generalization aspects. The proposed CCDM can exhibit superior forecasting capability compared to current state-of-the-art diffusion forecasters over a comprehensive benchmark, with best MSE and CRPS outcomes on $66.67\%$ and $83.33\%$ cases. Our code is publicly available at https://github.com/LSY-Cython/CCDM.

Figures (13)

• Figure 1: The schematic of proposed information-theoretic denoising-based contrastive diffusion learning. Bi-directional arrows indicate two learning ways are complementary. The bar chart depicts the average gains by contrastive diffusion refinement on diverse prediction setups for six datasets.
• Figure 2: The framework of denoising-based contrastive conditional diffusion forecaster.
• Figure 3: The diagram of channel-aware conditional denoiser architecture. Left: the whole network. Middle: channel-mixing DiT blocks. Right: channel-independent MLP dense modules.
• Figure 4: Comparison of generated point forecasts and prediction intervals on an Electricity channel.
• Figure 5: Forecasting results by varying contrastive weight $\lambda$ on three datasets with $H=168$.