UniTS: A Unified Multi-Task Time Series Model

TL;DR

UniTS tackles the challenge of unifying predictive and generative time series tasks within a single, multi-task model. It introduces task tokenization and a unified transformer backbone augmented with time- and variable-dimension self-attention, a dynamic FFN, gating, and shared GEN/CLS towers to support diverse tasks across heterogeneous domains. Through unified masked reconstruction pre-training and multi-task co-training, UniTS achieves state-of-the-art or competitive performance across 38 datasets, with strong few-shot and prompt-based adaptation, and direct multi-step forecasting without task-specific heads. The work demonstrates robust cross-domain transfer, efficient adaptation, and a path toward universal time series modeling with broad practical impact.

Abstract

Although pre-trained transformers and reprogrammed text-based LLMs have shown strong performance on time series tasks, the best-performing architectures vary widely across tasks, with most models narrowly focused on specific areas, such as time series forecasting. Unifying predictive and generative time series tasks within a single model remains challenging. We introduce UniTS, a unified multi-task time series model that utilizes task tokenization to integrate predictive and generative tasks into a single framework. UniTS employs a modified transformer block to capture universal time series representations, enabling transferability from a heterogeneous, multi-domain pre-training dataset-characterized by diverse dynamic patterns, sampling rates, and temporal scales-to a wide range of downstream datasets with varied task specifications and data domains. Tested on 38 datasets across human activity sensors, healthcare, engineering, and finance, UniTS achieves superior performance compared to 12 forecasting models, 20 classification models, 18 anomaly detection models, and 16 imputation models, including adapted text-based LLMs. UniTS also demonstrates strong few-shot and prompt capabilities when applied to new domains and tasks. In single-task settings, UniTS outperforms competitive task-specialized time series models. Code and datasets are available at https://github.com/mims-harvard/UniTS.

Figures (9)

• Figure 1: UniTS is a unified multi-task time series model for predictive and generative tasks.
• Figure 2: a) UniTS for forecasting; input is tokenized, and GEN tokens are un-patchified to infer the forecast horizon. b)UniTS for classification; a CLS token is used to represent class information and then compared to class tokens to get prediction class. c) Architecture of UniTS model.
• Figure 3: Direct multi-step forecasting on new lengths. UniTS achieves any new forecasting length with unified direct multi-step inference. Baseline methods use the sliding windows inference as they do not support direct multi-step inference.
• Figure 4: The network architecture of UniTS. Shared GEN tower and CLS tower transform task tokens to the prediction results of generative and predictive tasks.
• Figure 5: The dynamic FFN in UniTS.