CoRA: Covariate-Aware Adaptation of Time Series Foundation Models

TL;DR

This work tackles the mismatch between Time Series Foundation Models (TSFMs) and real-world forecasting tasks that involve diverse exogenous covariates. It presents CoRA, a covariate-aware adaptation framework that freezes pre-trained backbones and uses a Granger Causality Embedding to weight multi-modal covariates, injecting information through a zero-initialized adaLN module to preserve initialization. The approach achieves state-of-the-art results across uni-modal, multi-modal, few-shot, and multivariate forecasting, with substantial improvements such as a 31.1% reduction in MSE on covariate-aware tasks and notable gains on datasets like RT-1 and Time-MMD. CoRA generalizes across multiple TSFMs and modalities, offering a practical pathway to leverage large-scale foundation models for time-series forecasting while providing interpretable covariate causality insights and stable, scalable adaptation.

Abstract

Time Series Foundation Models (TSFMs) have shown significant impact through their model capacity, scalability, and zero-shot generalization. However, due to the heterogeneity of inter-variate dependencies and the backbone scalability on large-scale multivariate datasets, most TSFMs are typically pre-trained on univariate time series. This limitation renders them oblivious to crucial information from diverse covariates in real-world forecasting tasks. To further enhance the performance of TSFMs, we propose a general covariate-aware adaptation (CoRA) framework for TSFMs. It leverages pre-trained backbones of foundation models while effectively incorporating exogenous covariates from various modalities, including time series, language, and images, to improve the quality of predictions. Technically, CoRA maintains the equivalence of initialization and parameter consistency during adaptation. With preserved backbones of foundation models as frozen feature extractors, the outcome embeddings from foundation models are empirically demonstrated more informative than raw data. Further, CoRA employs a novel Granger Causality Embedding (GCE) to automatically evaluate covariates regarding their causal predictability with respect to the target variate. We incorporate these weighted embeddings with a zero-initialized condition-injection mechanism, avoiding catastrophic forgetting of pre-trained foundation models and gradually integrates exogenous information. Extensive experiments show that CoRA of TSFMs surpasses state-of-the-art covariate-aware deep forecasters with full or few-shot training samples, achieving 31.1% MSE reduction on covariate-aware forecasting. Compared to other adaptation methods, CoRA exhibits strong compatibility with various advanced TSFMs and extends the scope of covariates to other modalities, presenting a practical paradigm for the application of TSFMs.

Figures (9)

• Figure 1: CoRA performance on different covariate-aware benchmarks.
• Figure 2: Several paradigms of time series forecasting: (a) Covariate-aware deep models are supervisedly trained in a channel-dependent way. However, the backbone can be task-specific and challenged to scale up. (b) TSFMs designed to address data heterogeneity are generally pre-trained and predict on univariate time series. which makes them infeasible to utilize inter-variate dependencies explicitly. (c) CoRA leverages various foundation models, incorporates exogenous information to predict the target variate, and rapidly adapts to specific tasks without altering pre-trained models.
• Figure 3: Overall architecture of CoRA. CoRA freezes the backbone of foundation models as embedding extractors for multi-modal covariates, which are then selected by a trainable Granger Causality Embedding. This refined embedding is injected into the original TSFM head via a zero-initialized module to generate the shifting and scaling factors for final predictions.
• Figure 4: Multi-modal covariate-aware forecasting on a subset of RT-1 brohan2022rt with a time series target variate and an image covariate. Input length is set to 32 and prediction length is 4.
• Figure 5: Few-shot forecasting on the EPF dataset, comparing CoRA with TimeXer wang2024timexer and ChronosX arango2025chronosx across different levels of data availability.