Enhancing Zero-Shot Time Series Forecasting in Off-the-Shelf LLMs via Noise Injection

TL;DR

This work tackles the brittleness of zero-shot time series forecasting with off-the-shelf LLMs by introducing NLTS, an inference-time noise-injection approach that perturbs raw TS inputs before textualization and tokenization. The method leverages a fully frozen LLM, relying on robust temporal structure rather than numerical artifacts, and employs sampling with median aggregation to produce forecasts with quantified uncertainty. The authors provide a theoretical foundation showing perturbation stability and favorable Hessian properties for well-trained models, and validate NLTS across diverse benchmarks, including contamination-free synthetic and stock datasets, where it consistently improves forecasting accuracy without memorization. The results suggest NLTS as a practical, low-cost enhancement for deploying off-the-shelf LLMs in real-world TS tasks, with implications for robust, data-efficient forecasting in resource-constrained settings.

Abstract

Large Language Models (LLMs) have demonstrated effectiveness as zero-shot time series (TS) forecasters. The key challenge lies in tokenizing TS data into textual representations that align with LLMs' pre-trained knowledge. While existing work often relies on fine-tuning specialized modules to bridge this gap, a distinct, yet challenging, paradigm aims to leverage truly off-the-shelf LLMs without any fine-tuning whatsoever, relying solely on strategic tokenization of numerical sequences. The performance of these fully frozen models is acutely sensitive to the textual representation of the input data, as their parameters cannot adapt to distribution shifts. In this paper, we introduce a simple yet highly effective strategy to overcome this brittleness: injecting noise into the raw time series before tokenization. This non-invasive intervention acts as a form of inference-time augmentation, compelling the frozen LLM to extrapolate based on robust underlying temporal patterns rather than superficial numerical artifacts. We theoretically analyze this phenomenon and empirically validate its effectiveness across diverse benchmarks. Notably, to fully eliminate potential biases from data contamination during LLM pre-training, we introduce two novel TS datasets that fall outside all utilized LLMs' pre-training scopes, and consistently observe improved performance. This study provides a further step in directly leveraging off-the-shelf LLMs for time series forecasting.

Figures (11)

• Figure 1: Overview of zero-shot TS forecasting in off-the-shelf LLMs: the top is a vanilla usage of off-the-shelf LLM for TS, where the numerical values are tokenized and directly converted into a string, and then fed into a frozen LLM for prediction. The bottom is our NLTS framework, which introduces noise injection.
• Figure 2: Effect of noise level with NLTS. (a) and (b) illustrate predictions on the Traffic dataset under low and high noise levels. (c) summarizes the relative improvements of NLTS over the LLMTime baseline across multiple datasets under varying noise levels.
• Figure 3: Performance of our NLTS on ETTh2 under different noise types.
• Figure 4: Distribution comparison on the Traffic and ETTh2 dataset. Figures (a) and (d) show the empirical distributions of the training data before and after noise injection. Figures (b)-(c) illustrate test set output distributions on Traffic using LLMTime (without noise) and NLTS (with noise), across GPT-3.5-Turbo-Instruct and GLM-Air. Figures (e)-(f) show results on ETTh2 across Claude-3.5-Sonnet and GLM-Air. In all cases, NLTS outputs align more closely with the true data distribution, highlighting the robustness improvements from noise injection.
• Figure 5: Average zero-shot forecasting performance (MAE) across the Darts, Memorization, and Autoformer benchmarks.

Theorems & Definitions (4)

• Theorem 1: First- and second-order optimality for well-trained LLMs
• Lemma 1: Perturbation stability of well-trained LLMs
• proof
• proof