Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From Numbers to Prompts: A Cognitive Symbolic Transition Mechanism for Lightweight Time-Series Forecasting

Namkyung Yoon, Hwangnam Kim

TL;DR

The paper addresses the challenge of deploying accurate time-series forecasting on resource-constrained devices by introducing the Symbolic Transition Mechanism (STM), a lightweight wrapper that bridges numeric time-series data and language models through symbolic abstraction and prompt engineering. STM converts continuous series into a five-level symbolic representation, computes transition and periodicity cues, and generates an external attention descriptor that is injected into prompts without altering backbone model parameters. Across climate and traffic datasets, STM consistently improves MAE and MSE for multiple small language models, with gains up to 69% and 90% respectively, while incurring negligible increases in GPU memory and latency. The approach enables efficient, scalable time-series forecasting on edge devices and provides a practical pathway to integrate symbolic temporal priors into prompt-based forecasting pipelines. Future work includes expanding symbolic richness, testing on more domains, and exploring interactions with larger LMs beyond the current SLMs.

Abstract

Large language models have achieved remarkable success in time series prediction tasks, but their substantial computational and memory requirements limit deployment on lightweight platforms. In this paper, we propose the Symbolic Transition Mechanism (STM) a novel framework that bridges numeric time series data and language models through symbolic abstraction and prompt engineering. STM transforms continuous time series values into symbol tokens with quantization techniques based on human cognitive structures, and captures temporal dynamics through structured transformations of symbols, enabling fast engineering based predictions in which language models focus on critical parts of time series data. STM is a general purpose mechanisms that ensure the integrity of backbone language models, but they significantly improve their efficiency by inferring the dynamic and structured patterns inherent in time series data. We evaluated STM on various time series datasets, paired with four small language models (SLM) with limited computational environments. For all models, STM achieves error reductions of up to 69% in MAE and 90% in MSE compared to the default backbone SLM without STM. These results demonstrate the potential of STM as an efficient, adaptable layer for symbol-driven time series prediction using foundation models. The accuracy improvements were made at negligible resource costs, with maximum GPU memory of the base model increasing by approximately 0.06% and latency overhead increasing by only 0.64%.

From Numbers to Prompts: A Cognitive Symbolic Transition Mechanism for Lightweight Time-Series Forecasting

TL;DR

The paper addresses the challenge of deploying accurate time-series forecasting on resource-constrained devices by introducing the Symbolic Transition Mechanism (STM), a lightweight wrapper that bridges numeric time-series data and language models through symbolic abstraction and prompt engineering. STM converts continuous series into a five-level symbolic representation, computes transition and periodicity cues, and generates an external attention descriptor that is injected into prompts without altering backbone model parameters. Across climate and traffic datasets, STM consistently improves MAE and MSE for multiple small language models, with gains up to 69% and 90% respectively, while incurring negligible increases in GPU memory and latency. The approach enables efficient, scalable time-series forecasting on edge devices and provides a practical pathway to integrate symbolic temporal priors into prompt-based forecasting pipelines. Future work includes expanding symbolic richness, testing on more domains, and exploring interactions with larger LMs beyond the current SLMs.

Abstract

Large language models have achieved remarkable success in time series prediction tasks, but their substantial computational and memory requirements limit deployment on lightweight platforms. In this paper, we propose the Symbolic Transition Mechanism (STM) a novel framework that bridges numeric time series data and language models through symbolic abstraction and prompt engineering. STM transforms continuous time series values into symbol tokens with quantization techniques based on human cognitive structures, and captures temporal dynamics through structured transformations of symbols, enabling fast engineering based predictions in which language models focus on critical parts of time series data. STM is a general purpose mechanisms that ensure the integrity of backbone language models, but they significantly improve their efficiency by inferring the dynamic and structured patterns inherent in time series data. We evaluated STM on various time series datasets, paired with four small language models (SLM) with limited computational environments. For all models, STM achieves error reductions of up to 69% in MAE and 90% in MSE compared to the default backbone SLM without STM. These results demonstrate the potential of STM as an efficient, adaptable layer for symbol-driven time series prediction using foundation models. The accuracy improvements were made at negligible resource costs, with maximum GPU memory of the base model increasing by approximately 0.06% and latency overhead increasing by only 0.64%.
Paper Structure (23 sections, 6 equations, 5 figures, 3 tables)

This paper contains 23 sections, 6 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Language Model
  4. Small Language Model
  5. Attention mechanism
  6. Design Principle and Approach
  7. Theoretical Approach
  8. Symbolic Transition Mechanism
  9. Symbol-Based Encoding
  10. Transition-Based Weighting
  11. Periodicity Detection
  12. Attention Score
  13. Model Integration
  14. Experiments
  15. Data Acquisition and Pre-Processing
  16. ...and 8 more sections

Figures (5)

  • Figure 1: Overview of the proposed time series prediction model with the Symbolic Transition Mechanism.
  • Figure 2: The structure of Symbolic Transition Mechanism (Red box).
  • Figure 3: Validation with symbol ablation across four SLMs. Five-level encoding consistently provides optimal or near-optimal performance across models.
  • Figure 4: STM visualizations for climate temperature and LLM traffic datasets: (a,b) original time series, (c,d) symbolic-coded representations, and (e,f) STM-weighted symbolic encodings.
  • Figure 5: Effectiveness of STM on each model.