T-LLM: Teaching Large Language Models to Forecast Time Series via Temporal Distillation

TL;DR

The paper tackles the problem of equipping general-purpose LLMs with time-series forecasting capabilities without relying on extensive time-series pretraining. It introduces T-LLM, a temporal distillation framework where a lightweight temporal teacher, combining trend modeling and FFT-based frequency analysis, trains an LLM student through representation and prediction-level supervision; the teacher is removed at inference. Key contributions include the horizon-aware Dominant Spectral Projection (DSP), horizon-conditioned spectral capacity, and a head–tail guidance strategy for efficient temporal supervision, plus a comprehensive set of experiments showing strong performance in full-shot, few-shot, and zero-shot settings. The approach offers a practical, scalable alternative to scale-driven pretraining, enabling robust forecasting with a simple deployment pipeline in real-world domains like epidemiology.

Abstract

Time series forecasting plays a critical role in decision-making across many real-world applications. Unlike data in vision and language domains, time series data is inherently tied to the evolution of underlying processes and can only accumulate as real-world time progresses, limiting the effectiveness of scale-driven pretraining alone. This time-bound constraint poses a challenge for enabling large language models (LLMs) to acquire forecasting capability, as existing approaches primarily rely on representation-level alignment or inference-time temporal modules rather than explicitly teaching forecasting behavior to the LLM. We propose T-LLM, a temporal distillation framework that equips general-purpose LLMs with time series forecasting capability by transferring predictive behavior from a lightweight temporal teacher during training. The teacher combines trend modeling and frequency-domain analysis to provide structured temporal supervision, and is removed entirely at inference, leaving the LLM as the sole forecasting model. Experiments on benchmark datasets and infectious disease forecasting tasks demonstrate that T-LLM consistently outperforms existing LLM-based forecasting methods under full-shot, few-shot, and zero-shot settings, while enabling a simple and efficient deployment pipeline.

Figures (6)

• Figure 1: The Time-Bound Constraint. CV/NLP data can be collected independently of real-world time, whereas time-series data grows only as real processes evolve, creating a "Future Wall" for scale-driven pretraining.
• Figure 2: Overview of the proposed temporal distillation framework with a training-only temporal teacher and an inference-ready LLM student.
• Figure 3: Ablation of head–tail guidance compared to full layer-wise guidance.
• Figure 4: Ablation of DSP capacity choices ($d_{\text{red}}\in\{32,64,96,128\}$) across different prediction horizons.
• Figure 5: Visualization of attention centered kernel alignment (CKA) similarity across datasets. The y-axis denotes transformer layers, and the x-axis denotes training epochs.