LAST SToP For Modeling Asynchronous Time Series

TL;DR

LAST SToP for Modeling Asynchronous Time Series presents LASTS, a prompt-based framework that enables LLMs to process asynchronous time series described as natural language events. It introduces StoP to enable parameter-efficient adaptation and demonstrates state-of-the-art performance on forecasting, anomaly detection, and imputation across real-world textual and TPP datasets. LASTS uses a three-part prompt structure and explores representation and PEFT choices, achieving better generalization and training efficiency than SP and QLoRA baselines. The work highlights the potential of LLM-based representations for asynchronous time series and suggests future directions in hybrid architectures and broader modalities.

Abstract

We present a novel prompt design for Large Language Models (LLMs) tailored to Asynchronous Time Series. Unlike regular time series, which assume values at evenly spaced time points, asynchronous time series consist of timestamped events occurring at irregular intervals, each described in natural language. Our approach effectively utilizes the rich natural language of event descriptions, allowing LLMs to benefit from their broad world knowledge for reasoning across different domains and tasks. This allows us to extend the scope of asynchronous time series analysis beyond forecasting to include tasks like anomaly detection and data imputation. We further introduce Stochastic Soft Prompting, a novel prompt-tuning mechanism that significantly improves model performance, outperforming existing fine-tuning methods such as QLoRA. Through extensive experiments on real world datasets, we demonstrate that our approach achieves state-of-the-art performance across different tasks and datasets.

Figures (13)

• Figure 1: We show that our LASTS framework can solve the following tasks on asynchronous time series data: (a)Forecasting:(top) The model is given a sequence of events, encoded as text, with the goal of predicting the next event. (b)Anomaly detection:(middle) The model is given a sequence of events containing an incorrect event (bold) with the goal of finding the incorrect event. (c)Imputation: (bottom) The model is given a sequence of events containing a masked event, encoded as text, with the goal of predicting the masked event.
• Figure 2: Components of a LASTS prompt: A concise task description is included in the system prompt, while asynchronous time series is provided as an input in the user prompt.
• Figure 3: Comparison of Soft Prompt (SP) and Stochastic Soft Prompt (StoP) training. For illustration, the soft prompt $P$ is of length $50$. In SP, the entire prompt is used during both training and inference. In StoP, a random prefix of $P$ is used per training batch, while the full prompt is used for inference. Fire marks the soft prompt, which is the trainable prompt portion, while snowflake represents the frozen LASTS text prompt.
• Figure 4: Macro-F1 $\uparrow$, MAE $\downarrow$, and Accuracy $\uparrow$, averaged across all datasets for Forecast and Imputation for Zero Shot methods.
• Figure 5: Learned token representations of StoP and SP. The first two plots show t-SNE projections of 100 tokens from 400-length prompts (Breakfast dataset, forecasting)—StoP tokens are more dispersed, while SP tokens cluster closely. The third plot shows lower adjacent token cosine similarity for StoP (blue) than SP (red), indicating greater diversity.