Augmenting LLMs for General Time Series Understanding and Prediction

TL;DR

TsLLM introduces a patch-based time-series encoder-decoder that feeds a pretrained decoder-only LLM through adapters, enabling interleaved text and time-series inputs and outputs. By employing a $eta$-VAE objective and scale-shape decomposition, it learns high-fidelity temporal representations and preserves numerical fidelity while maintaining language abilities. A three-stage training curriculum aligns modalities and broadens capabilities to contextual forecasting, time-series QA, and narrative report generation across diverse domains, achieving strong results on ECG-QA, contextual forecasting, and cross-task transfer, while remaining competitive on traditional forecasting benchmarks. The work democratizes temporal reasoning by allowing domain experts to interact with time-series data via natural language, though it acknowledges computational costs and remaining gaps in pure numerical-task performance and multivariate modeling.

Abstract

Time series data is fundamental to decision-making in many crucial domains including healthcare, finance, and environmental science. However, analyzing this data often requires incorporating unstructured contextual information, answering domain-specific questions, and generating natural language explanations -- capabilities that traditional time series models lack due to their inability to process text. While Large Language Models (LLMs) excel at contextual reasoning and knowledge integration, they struggle with numerical time series due to inefficient text-based representations and limited exposure to temporal data during pretraining. We address this gap by augmenting an LLM with specialized time series perception through a patch-based encoder-decoder architecture. We train this Time Series-augmented LLM (TsLLM) on a large corpus of over 2 million interleaved time series and text examples spanning diverse analysis tasks: forecasting with contextual information, time series question-answering, pattern explanation, classification with natural language outputs, and report generation. This training enables TsLLM to leverage both its language understanding and newly acquired temporal reasoning capabilities. While not designed to surpass specialized models on traditional benchmarks, TsLLM demonstrates strong performance on tasks requiring the integration of time series analysis with natural language -- capabilities that existing approaches cannot provide. Our work establishes a new paradigm for time series analysis that bridges numerical computation and natural language understanding, democratizing access to sophisticated temporal reasoning through natural language interaction.

Figures (3)

• Figure 1: TsLLM architecture enabling novel interleaving of text (blue) and time series (green) inputs and outputs. The time series encoder-decoder has been pretrained to produce high-quality time series patch representations, which our multi-stage alignment process (snowflake = frozen, fire = trained) bridges with the LLM to enable unified text and time series processing.
• Figure 2: Illustrative examples of our unified framework where time series data is integrated with the text in both the inputs and outputs. This figure visualizes these embedded time series segments as plots for illustrative clarity. Examples demonstrate versatility across tasks like: (Top) contextual forecasting (e.g., ECG signals), (Middle) classification (e.g., sensor data), and (Bottom) report generation (e.g., COVID mobility).
• Figure 3: The relationship between patch size for the time series encoder and reconstruction MSE for the VAE on the validation set.