Time-LLM: Time Series Forecasting by Reprogramming Large Language Models

TL;DR

The paper introduces Time-LLM, a reprogramming framework that repurposes frozen large language models for general time series forecasting by converting time-series patches into text prototypes and guiding their transformation with Prompt-as-Prefix prompts. It achieves state-of-the-art results across long- and short-term forecasting, and excels in few-shot and zero-shot settings without fine-tuning the backbone model. The approach emphasizes cross-modality alignment, data efficiency, and leveraging LLM reasoning for time-series tasks, pointing toward multimodal foundation models that can handle language and sequential data. It also demonstrates substantial efficiency advantages over traditional fine-tuning, making time-series forecasting more generalizable and scalable.

Abstract

Time series forecasting holds significant importance in many real-world dynamic systems and has been extensively studied. Unlike natural language process (NLP) and computer vision (CV), where a single large model can tackle multiple tasks, models for time series forecasting are often specialized, necessitating distinct designs for different tasks and applications. While pre-trained foundation models have made impressive strides in NLP and CV, their development in time series domains has been constrained by data sparsity. Recent studies have revealed that large language models (LLMs) possess robust pattern recognition and reasoning abilities over complex sequences of tokens. However, the challenge remains in effectively aligning the modalities of time series data and natural language to leverage these capabilities. In this work, we present Time-LLM, a reprogramming framework to repurpose LLMs for general time series forecasting with the backbone language models kept intact. We begin by reprogramming the input time series with text prototypes before feeding it into the frozen LLM to align the two modalities. To augment the LLM's ability to reason with time series data, we propose Prompt-as-Prefix (PaP), which enriches the input context and directs the transformation of reprogrammed input patches. The transformed time series patches from the LLM are finally projected to obtain the forecasts. Our comprehensive evaluations demonstrate that Time-LLM is a powerful time series learner that outperforms state-of-the-art, specialized forecasting models. Moreover, Time-LLM excels in both few-shot and zero-shot learning scenarios.

Figures (10)

• Figure 1: Schematic illustration of reprogramming large language models (LLMs) in comparison of (a) task-specific learning and (b) model fine-tuning. Our proposal investigates and demonstrates (c) how to effectively reprogram open-sourced LLMs as powerful time series learners where well-developed time series pre-trained models are not readily available.
• Figure 2: The model framework of Time-LLM. Given an input time series, we first tokenize and embed it via [mycircled,red]a11 patching along with a [mycircled,red]a22 customized embedding layer. [mycircled,red]a33 These patch embeddings are then reprogrammed with condensed text prototypes to align two modalities. To augment the LLM's reasoning ability, [mycircled,red]a44 additional prompt prefixes are added to the input to direct the transformation of input patches. [mycircled,red]a55 The output patches from the LLM are projected to generate the forecasts.
• Figure 3: Illustration of (a) patch reprogramming and (b) Patch-as-Prefix versus Prompt-as-Prefix.
• Figure 4: Prompt example. $<>$ and $<>$ are task-specific configurations and calculated input statistics.
• Figure 5: A showcase of patch reprogramming.