Towards Cross-Modality Modeling for Time Series Analytics: A Survey in the LLM Era

TL;DR

This survey addresses the cross-modality gap between time series data and textual representations in the LLM era by proposing a textual-data-centric taxonomy that partitions textual inputs into numerical prompts, statistical prompts, contextual prompts, and word token embeddings. It formalizes the cross-modality objective as $f:(oldsymbol{X},\boldsymbol{T})\rightarrow \boldsymbol{Y}$ and organizes methods into alignment (retrieval, contrastive learning, knowledge distillation) and fusion (addition, concatenation) strategies. Through multi-domain experiments on five real-world datasets, the study shows that textual data—especially numerical prompts—and retrieval-based alignment substantially boost forecasting performance, with domain-specific effects outlined. The paper also discusses future directions in multi-modality integration, efficiency, and transparency, providing actionable guidance for researchers and practitioners developing LLM-based time series analytics.

Abstract

The proliferation of edge devices has generated an unprecedented volume of time series data across different domains, motivating various well-customized methods. Recently, Large Language Models (LLMs) have emerged as a new paradigm for time series analytics by leveraging the shared sequential nature of textual data and time series. However, a fundamental cross-modality gap between time series and LLMs exists, as LLMs are pre-trained on textual corpora and are not inherently optimized for time series. Many recent proposals are designed to address this issue. In this survey, we provide an up-to-date overview of LLMs-based cross-modality modeling for time series analytics. We first introduce a taxonomy that classifies existing approaches into four groups based on the type of textual data employed for time series modeling. We then summarize key cross-modality strategies, e.g., alignment and fusion, and discuss their applications across a range of downstream tasks. Furthermore, we conduct experiments on multimodal datasets from different application domains to investigate effective combinations of textual data and cross-modality strategies for enhancing time series analytics. Finally, we suggest several promising directions for future research. This survey is designed for a range of professionals, researchers, and practitioners interested in LLM-based time series modeling.

Figures (4)

• Figure 1: Cross-Modality Modeling for Time Series Analytics.
• Figure 2: Taxonomy of cross-modality modeling for time series (TS) analytics incorporating textual data, including numerical prompt, statistical prompt, contextual prompt, and word token embedding. The textual data is processed by LLMs.
• Figure 3: Distribution of taxonomies in cross-modality time series modeling. (a) Textual data types: numerical prompts $\mathbf P_N$, statistical prompts $\mathbf P_S$, contextual prompts $\mathbf P_C$, and word token embeddings $\mathbf{W}$. (b) Strategy: Alignment vs. Fusion. (c) Method categories: unidirectional retrieval (UDR), bidirectional retrieval (BDR), contrastive learning (CL), knowledge distillation (KD), concatenation (Concat), and addition (Add). (d) Task types: forecasting, classification, anomaly detection, and multiple tasks.
• Figure 4: Overall Framework.