CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting

TL;DR

CoST advocates learning disentangled seasonal-trend representations for time series forecasting via contrastive learning. It introduces a dual-path framework: a Trend Feature Disentangler using a mixture of auto-regressive experts and a Seasonal Feature Disentangler using a learnable Fourier layer, coupled with time- and frequency-domain contrastive losses. The approach achieves state-of-the-art MSE improvements across multivariate and univariate benchmarks and demonstrates robustness to backbone encoders and downstream regressors. The work provides a causal interpretation for disentanglement, a principled loss architecture, and extensive ablations and case studies validating the benefits of separating seasonal and trend information before regression. This representation-first paradigm has practical implications for generalization in non-stationary time-series environments.

Abstract

Deep learning has been actively studied for time series forecasting, and the mainstream paradigm is based on the end-to-end training of neural network architectures, ranging from classical LSTM/RNNs to more recent TCNs and Transformers. Motivated by the recent success of representation learning in computer vision and natural language processing, we argue that a more promising paradigm for time series forecasting, is to first learn disentangled feature representations, followed by a simple regression fine-tuning step -- we justify such a paradigm from a causal perspective. Following this principle, we propose a new time series representation learning framework for time series forecasting named CoST, which applies contrastive learning methods to learn disentangled seasonal-trend representations. CoST comprises both time domain and frequency domain contrastive losses to learn discriminative trend and seasonal representations, respectively. Extensive experiments on real-world datasets show that CoST consistently outperforms the state-of-the-art methods by a considerable margin, achieving a 21.3% improvement in MSE on multivariate benchmarks. It is also robust to various choices of backbone encoders, as well as downstream regressors. Code is available at https://github.com/salesforce/CoST.

Figures (5)

• Figure 1: Time series composed of seasonal and trend components.
• Figure 2: Causal graph of the generative process for time series data.
• Figure 3: (a) Overall Framework. Given intermediate representations from the backbone encoder, $\tilde{{\bm{V}}} = f_b({\bm{X}})$, the TFD and SFD produce the trend features, ${\bm{V}}^{(T)} = f_T(\tilde{{\bm{V}}})$, and seasonal features, ${\bm{V}}^{(S)} = f_S(\tilde{{\bm{V}}})$, respectively. (b) Trend Feature Disentangler. Composition of a mixture of auto-regressive experts, instantiated as 1d-causal convolutions with kernel size of $2^i, \forall i = 0, \ldots, L$, where $L$ is a hyper-parameter. Followed by average-pool over the $L+1$ representations. (c) Seasonal Feature Disentangler. After transforming the intermediate representations into frequency domain via the FFT, the SFD applies a (complex-valued) linear layer with unique weights for each frequency. Then, an inverse FFT is performed to map the representations back to time domain, to form the seasonal representations, ${\bm{V}}^{(S)}$.
• Figure 4: T-SNE visualization of learned representations from CoST and TS2Vec. (Top) Generated by visualizing representations after selecting a single seasonality. Colors represent the two distinct trends. (Bottom) Generated by visualizing representations after selecting a single trend. Colors represent the three distinct seasonal patterns.
• Figure 5: T-SNE visualization of seasonal-trend disentanglement in CoST embeddings. TFD Representations refer to the representations generated by the Trend Feature Disentangler while SFD Representations refer to the representations generated by the Seasonal Feature Disentangler. (a) We select a single seasonality and visualize the representations. The two colors represent the two distinct trends. (b) We select a single trend and visualize the representations. The three colors represent the three distinct seasonal patterns.