Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

StockBot 2.0: Vanilla LSTMs Outperform Transformer-based Forecasting for Stock Prices

Shaswat Mohanty

TL;DR

This paper tackles stock price forecasting by systematically comparing diverse temporal architectures (vanilla LSTM, attention-based LSTMs, TCNs, Informer, TFT) within a unified StockBot framework. Using Yahoo Finance data from 2010–2020 and a consistent default-hyperparameter setup, it evaluates both predictive accuracy and downstream trading performance under one-day-ahead and ten-day-ahead horizons. The key finding is that a carefully configured vanilla LSTM consistently outperforms transformer-based architectures in both accuracy and trading stability in data-limited settings. This underscores the importance of architectural inductive bias and data efficiency for practical financial time-series forecasting and suggests focusing on robust, simple models or targeted hyperparameter optimization for real-world trading applications.

Abstract

Accurate forecasting of financial markets remains a long-standing challenge due to complex temporal and often latent dependencies, non-linear dynamics, and high volatility. Building on our earlier recurrent neural network framework, we present an enhanced StockBot architecture that systematically evaluates modern attention-based, convolutional, and recurrent time-series forecasting models within a unified experimental setting. While attention-based and transformer-inspired models offer increased modeling flexibility, extensive empirical evaluation reveals that a carefully constructed vanilla LSTM consistently achieves superior predictive accuracy and more stable buy/sell decision-making when trained under a common set of default hyperparameters. These results highlight the robustness and data efficiency of recurrent sequence models for financial time-series forecasting, particularly in the absence of extensive hyperparameter tuning or the availability of sufficient data when discretized to single-day intervals. Additionally, these results underscore the importance of architectural inductive bias in data-limited market prediction tasks.

StockBot 2.0: Vanilla LSTMs Outperform Transformer-based Forecasting for Stock Prices

TL;DR

This paper tackles stock price forecasting by systematically comparing diverse temporal architectures (vanilla LSTM, attention-based LSTMs, TCNs, Informer, TFT) within a unified StockBot framework. Using Yahoo Finance data from 2010–2020 and a consistent default-hyperparameter setup, it evaluates both predictive accuracy and downstream trading performance under one-day-ahead and ten-day-ahead horizons. The key finding is that a carefully configured vanilla LSTM consistently outperforms transformer-based architectures in both accuracy and trading stability in data-limited settings. This underscores the importance of architectural inductive bias and data efficiency for practical financial time-series forecasting and suggests focusing on robust, simple models or targeted hyperparameter optimization for real-world trading applications.

Abstract

Accurate forecasting of financial markets remains a long-standing challenge due to complex temporal and often latent dependencies, non-linear dynamics, and high volatility. Building on our earlier recurrent neural network framework, we present an enhanced StockBot architecture that systematically evaluates modern attention-based, convolutional, and recurrent time-series forecasting models within a unified experimental setting. While attention-based and transformer-inspired models offer increased modeling flexibility, extensive empirical evaluation reveals that a carefully constructed vanilla LSTM consistently achieves superior predictive accuracy and more stable buy/sell decision-making when trained under a common set of default hyperparameters. These results highlight the robustness and data efficiency of recurrent sequence models for financial time-series forecasting, particularly in the absence of extensive hyperparameter tuning or the availability of sufficient data when discretized to single-day intervals. Additionally, these results underscore the importance of architectural inductive bias in data-limited market prediction tasks.
Paper Structure (14 sections, 26 equations, 5 figures, 2 tables, 1 algorithm)

This paper contains 14 sections, 26 equations, 5 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Data and Methods
  3. Data Preparation
  4. Model Architectures
  5. LSTM and Attention-Enhanced LSTM
  6. Temporal Convolutional Network (TCN)
  7. Informer Variant
  8. Temporal Fusion Transformer (TFT)
  9. Training and Hyperparameters
  10. Stock predicting bot
  11. Results
  12. One-Day-Ahead Forecasting
  13. Ten-Day-Ahead Forecasting
  14. Conclusions

Figures (5)

  • Figure 1: Generic temporal forecasting framework shared by all models considered in this work. Past observations are encoded into a latent temporal representation, which is mapped to future predictions. The dashed loop indicates optional autoregressive forecasting for multi-step prediction.
  • Figure 2: One-day-ahead forecasting results for AAPL. Top-left: autoregressive price prediction. Top-right: teacher-forced price prediction. Bottom-left: portfolio value driven by autoregressive forecasts. Bottom-right: portfolio value driven by teacher-forced forecasts.
  • Figure 3: One-day-ahead forecasting results for MSFT. Top-left: autoregressive price prediction. Top-right: teacher-forced price prediction. Bottom-left: portfolio value driven by autoregressive forecasts. Bottom-right: portfolio value driven by teacher-forced forecasts.
  • Figure 4: Ten-day-ahead forecasting results for AAPL. Top-left: autoregressive price prediction. Top-right: teacher-forced price prediction. Bottom-left: portfolio value driven by autoregressive forecasts. Bottom-right: portfolio value driven by teacher-forced forecasts.
  • Figure 5: Ten-day-ahead forecasting results for MSFT. Top-left: autoregressive price prediction. Top-right: teacher-forced price prediction. Bottom-left: portfolio value driven by autoregressive forecasts. Bottom-right: portfolio value driven by teacher-forced forecasts.