UniCA: Unified Covariate Adaptation for Time Series Foundation Model

Abstract

Time Series Foundation Models (TSFMs) have achieved remarkable success through large-scale pretraining. However, their design primarily targets real-valued series, limiting their ability to handle general forecasting tasks involving diverse and often heterogeneous covariates -- such as categorical variables and multimodal data (e.g., images, text) -- which are typically task-specific and difficult to leverage during pretraining. To address this gap, we propose Unified Covariate Adaptation (UniCA), a framework to bridge TSFMs with general covariate-aware forecasting. UniCA first performs covariate homogenization to transform heterogeneous covariates into high-level homogeneous series representations and then fuses them via a unified attention-based fusion mechanism. UniCA is compatible and universal for adaptation with both homogeneous and heterogeneous covariates, incorporating extra covariate information while preserving the generalization ability of TSFMs.Extensive experiments on multiple unimodal and multimodal covariate-aware forecasting benchmarks demonstrate the superiority of UniCA, highlighting the promise of covariate-aware TSFM adaptation in real-world forecasting scenarios.Code: https://github.com/hanlu-nju/UniCA.

Figures (13)

• Figure 1: TSFMs are pretrained on time series from diverse domains. However, many tasks contain homo/heterogeneous covariates that are hard to use in pre-training. Adaptation methods to handle these covariates are important in these tasks.
• Figure 2: Overview of Unified Covariate Adapter (UniCA). UniCA consists of two key pipelines (1) Covariate Homogenization: We use a converter to transform heterogeneous covariates into dense continuous series representations, thus reducing the heterogeneity gap between covariates and target time series. (2) Modular Fusion: We decompose the TSFM architecture into interpretable stages and insert Pre-Fusion and Post-Fusion modules to inject covariate information at appropriate locations without interfering with the model's pretrained dynamics.
• Figure 3: Forecasting performance on general covariate-aware forecasting datasets, including 12 unimodal datasets and multi-modal datasets MMSP and Time-MMD. Results are reported as MAPE averaged over sub-datasets for both unimodal and Time-MMD datasets. For the MMSP dataset, MAE is used instead, as near-zero target values render MAPE unstable.
• Figure 4: Average relative MAPE on unimodal datasets with model setups (a) Pretrained: Moirai(ZS), Adapted: Moirai (UniCA). (b) Finetuned: Chronos-Bolt (UniCA) with fine-tuned backbone, Frozen:freeze backbone; Ablation on (c) structure of covariate homogenizer. (d) hidden dimension of covariate homogenizer.
• Figure 5: Analysis of UniCA. (a) Efficiency on Time-MMD: The adapter adds minimal overhead in inference time (left panel) and trainable parameters (right panel). (b) Covariate Homogenization on MMSP: Aligned heterogeneous covariates reveal meaningful patterns like seasonality and trends. (c) Attention Maps: The fusion module dynamically attends to different covariates over time for the sample in (b).