Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Impact of LLMs news Sentiment Analysis on Stock Price Movement Prediction

Walid Siala, Ahmed Khanfir, Mike Papadakis

TL;DR

This paper tackles stock price movement prediction by leveraging LLM-based news sentiment analysis, addressing the lack of cross-model comparisons and representation analyses. It compares FinBERT, RoBERTa, and DeBERTa for financial sentiment, integrating their outputs with state-of-the-art time-series models (LSTM, PatchTST, TimesNet, tPatchGNN) to predict stock movements for five major equities over a multi-year period. The findings show DeBERTa achieves the highest per-news sentiment accuracy (~75%), and an ensemble of the three models via SVM reaches about 80% accuracy, with sentiment features providing modest gains to some architectures, particularly in classification and regression tasks. The results offer practical insights into combining LLM-based sentiment with advanced time-series models, highlighting both the complementarity of sentiment models and the limited but tangible benefits for certain stock-prediction tasks, all within a reproducible experimental framework.

Abstract

This paper addresses stock price movement prediction by leveraging LLM-based news sentiment analysis. Earlier works have largely focused on proposing and assessing sentiment analysis models and stock movement prediction methods, however, separately. Although promising results have been achieved, a clear and in-depth understanding of the benefit of the news sentiment to this task, as well as a comprehensive assessment of different architecture types in this context, is still lacking. Herein, we conduct an evaluation study that compares 3 different LLMs, namely, DeBERTa, RoBERTa and FinBERT, for sentiment-driven stock prediction. Our results suggest that DeBERTa outperforms the other two models with an accuracy of 75% and that an ensemble model that combines the three models can increase the accuracy to about 80%. Also, we see that sentiment news features can benefit (slightly) some stock market prediction models, i.e., LSTM-, PatchTST- and tPatchGNN-based classifiers and PatchTST- and TimesNet-based regression tasks models.

Impact of LLMs news Sentiment Analysis on Stock Price Movement Prediction

TL;DR

This paper tackles stock price movement prediction by leveraging LLM-based news sentiment analysis, addressing the lack of cross-model comparisons and representation analyses. It compares FinBERT, RoBERTa, and DeBERTa for financial sentiment, integrating their outputs with state-of-the-art time-series models (LSTM, PatchTST, TimesNet, tPatchGNN) to predict stock movements for five major equities over a multi-year period. The findings show DeBERTa achieves the highest per-news sentiment accuracy (~75%), and an ensemble of the three models via SVM reaches about 80% accuracy, with sentiment features providing modest gains to some architectures, particularly in classification and regression tasks. The results offer practical insights into combining LLM-based sentiment with advanced time-series models, highlighting both the complementarity of sentiment models and the limited but tangible benefits for certain stock-prediction tasks, all within a reproducible experimental framework.

Abstract

This paper addresses stock price movement prediction by leveraging LLM-based news sentiment analysis. Earlier works have largely focused on proposing and assessing sentiment analysis models and stock movement prediction methods, however, separately. Although promising results have been achieved, a clear and in-depth understanding of the benefit of the news sentiment to this task, as well as a comprehensive assessment of different architecture types in this context, is still lacking. Herein, we conduct an evaluation study that compares 3 different LLMs, namely, DeBERTa, RoBERTa and FinBERT, for sentiment-driven stock prediction. Our results suggest that DeBERTa outperforms the other two models with an accuracy of 75% and that an ensemble model that combines the three models can increase the accuracy to about 80%. Also, we see that sentiment news features can benefit (slightly) some stock market prediction models, i.e., LSTM-, PatchTST- and tPatchGNN-based classifiers and PatchTST- and TimesNet-based regression tasks models.
Paper Structure (13 sections, 1 figure, 2 tables)

This paper contains 13 sections, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Proposed Experimental Study
  3. Sentiment analysis models:
  4. Stock prediction methods:
  5. Datasets:
  6. Proposed daily sentiment representations:
  7. The prediction targets
  8. Metrics:
  9. Implementation details:
  10. Results and discussion
  11. Effectiveness and complementarity of sentiment analysis models
  12. Impact of news sentiment on stock movement prediction
  13. Conclusion

Figures (1)

  • Figure 1: Venn diagrams showing overlap among Original labels, FinBERT, RoBERTa, and DeBERTa across sentiment classes.