Time-Prompt: Integrated Heterogeneous Prompts for Unlocking LLMs in Time Series Forecasting

TL;DR

Time-Prompt introduces an LLM-empowered framework for time-series forecasting that activates large language models through dual-path prompting, semantic-space embedding, and cross-modal alignment, followed by LoRA-based fine-tuning to handle continuous values. The approach reprograms time-series data into textual-like representations, fuses text and temporal information, and guides the LLM with both soft and hard prompts to improve forecasting accuracy. Across nine datasets including carbon-emission data, Time-Prompt consistently outperforms strong SOTAs in long-term and few-shot settings, with ablation analyses highlighting the importance of CMA and LoRA. The work demonstrates that carefully designed prompting and cross-modal fusion can realize practical, scalable improvements in time-series forecasting and points toward future direction with time-series foundation models and smaller LLMs.

Abstract

Time series forecasting aims to model temporal dependencies among variables for future state inference, holding significant importance and widespread applications in real-world scenarios. Although deep learning-based methods have achieved remarkable progress, they still exhibit suboptimal performance in long-term forecasting. Recent research demonstrates that large language models (LLMs) achieve promising performance in time series forecasting, but this progress is still met with skepticism about whether LLMs are truly useful for this task. To address this, we propose Time-Prompt, a framework for activating LLMs for time series forecasting. Specifically, we first construct a unified prompt paradigm with learnable soft prompts to guide the LLM's behavior and textualized hard prompts to enhance the time series representations. Second, to enhance LLM' comprehensive understanding of the forecasting task, we design a semantic space embedding and cross-modal alignment module to achieve fusion of temporal and textual data. Finally, we efficiently fine-tune the LLM's parameters using time series data. Furthermore, we focus on carbon emissions, aiming to provide a modest contribution to global carbon neutrality. Comprehensive evaluations on 6 public datasets and 3 carbon emission datasets demonstrate that Time-Prompt is a powerful framework for time series forecasting.

Figures (3)

• Figure 1: a) Time series-based LLMs: Lack textual information, failing to leverage LLMs' potential for time series forecasting. (b) Prompt-based LLMs: Struggle to capture fine-grained temporal patterns from textual prompts. (c) Multimodal-based LLMs: Time series are processed via an encoder, and text via LLMs, separately. (d) Multimodal-based LLMs: Time series and text information are processed by LLMs in a unified manner.
• Figure 2: Time-Prompt framework. First, hard prompts and soft prompts are constructed based on time series. Second, the time series and prompts are embedded into the LLM's semantic space, followed by cross-modal alignment. Finally, the fused information is fed into the pretrained LLM and projected through output layers to generate forecasting results.
• Figure 3: Hard prompt example on chinaCarbon dataset.