Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

StockTime: A Time Series Specialized Large Language Model Architecture for Stock Price Prediction

Shengkun Wang, Taoran Ji, Linhan Wang, Yanshen Sun, Shang-Ching Liu, Amit Kumar, Chang-Tien Lu

TL;DR

StockTime is introduced, a novel LLM-based architecture designed specifically for stock price data that outperforms recent LLMs, as it gives more accurate predictions while reducing memory usage and runtime costs.

Abstract

The stock price prediction task holds a significant role in the financial domain and has been studied for a long time. Recently, large language models (LLMs) have brought new ways to improve these predictions. While recent financial large language models (FinLLMs) have shown considerable progress in financial NLP tasks compared to smaller pre-trained language models (PLMs), challenges persist in stock price forecasting. Firstly, effectively integrating the modalities of time series data and natural language to fully leverage these capabilities remains complex. Secondly, FinLLMs focus more on analysis and interpretability, which can overlook the essential features of time series data. Moreover, due to the abundance of false and redundant information in financial markets, models often produce less accurate predictions when faced with such input data. In this paper, we introduce StockTime, a novel LLM-based architecture designed specifically for stock price data. Unlike recent FinLLMs, StockTime is specifically designed for stock price time series data. It leverages the natural ability of LLMs to predict the next token by treating stock prices as consecutive tokens, extracting textual information such as stock correlations, statistical trends and timestamps directly from these stock prices. StockTime then integrates both textual and time series data into the embedding space. By fusing this multimodal data, StockTime effectively predicts stock prices across arbitrary look-back periods. Our experiments demonstrate that StockTime outperforms recent LLMs, as it gives more accurate predictions while reducing memory usage and runtime costs.

StockTime: A Time Series Specialized Large Language Model Architecture for Stock Price Prediction

TL;DR

StockTime is introduced, a novel LLM-based architecture designed specifically for stock price data that outperforms recent LLMs, as it gives more accurate predictions while reducing memory usage and runtime costs.

Abstract

The stock price prediction task holds a significant role in the financial domain and has been studied for a long time. Recently, large language models (LLMs) have brought new ways to improve these predictions. While recent financial large language models (FinLLMs) have shown considerable progress in financial NLP tasks compared to smaller pre-trained language models (PLMs), challenges persist in stock price forecasting. Firstly, effectively integrating the modalities of time series data and natural language to fully leverage these capabilities remains complex. Secondly, FinLLMs focus more on analysis and interpretability, which can overlook the essential features of time series data. Moreover, due to the abundance of false and redundant information in financial markets, models often produce less accurate predictions when faced with such input data. In this paper, we introduce StockTime, a novel LLM-based architecture designed specifically for stock price data. Unlike recent FinLLMs, StockTime is specifically designed for stock price time series data. It leverages the natural ability of LLMs to predict the next token by treating stock prices as consecutive tokens, extracting textual information such as stock correlations, statistical trends and timestamps directly from these stock prices. StockTime then integrates both textual and time series data into the embedding space. By fusing this multimodal data, StockTime effectively predicts stock prices across arbitrary look-back periods. Our experiments demonstrate that StockTime outperforms recent LLMs, as it gives more accurate predictions while reducing memory usage and runtime costs.
Paper Structure (19 sections, 5 equations, 3 figures, 4 tables)

This paper contains 19 sections, 5 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Stock Prediction
  4. Financial LLMs
  5. LLMs for Time Series
  6. Methodology
  7. Problem Definition
  8. Stocktime Overview
  9. Patched input
  10. Autoregressive Encoder
  11. Multimodal Fusion
  12. Prediction
  13. Experiments
  14. Experimental Setup
  15. Overall Performance and Analysis
  16. ...and 4 more sections

Figures (3)

  • Figure 1: The framework of existing FinLLMs. By applying instruction fine-tuning to general LLMs, FinLLMs update their model parameters. Then, they use different prompts to address various downstream tasks.
  • Figure 2: The StockTime framework operates as follows: (1) Stock correlations, statistical trends, and time step information are extracted from stock prices and processed as textual information through a frozen LLM. (2) Stock time series data is segmented and embedded, then passed through an autoregressive encoder to be integrated with textual information, which is subsequently processed by a pre-trained LLM. (3) After learning the multimodal information, the off the-shelf LLM as an autoregressive forecaster to predict the next token, which corresponds to the predicted stock price.
  • Figure 3: Hyperparameter sensitivity analysis of different lookback window lengths and autoregressive encoder layers.