Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BERTopic-Driven Stock Market Predictions: Unraveling Sentiment Insights

Enmin Zhu, Jerome Yen

TL;DR

This work tackles stock price prediction by integrating investor sentiment extracted from online stock discussions with deep learning models. It introduces BERTopic as a topic-modeling backbone to derive topic-level sentiment from stock-related text and evaluates its impact when fused with LSTM, CNN, GAN, and CNN-LSTM architectures. Across experiments using BERT- and VADER-based sentiment signals, topic-enhanced predictions consistently improve metrics such as RMSE, MAE, and R2, demonstrating that topic-level insights contain valuable information about volatility and price trends. The study suggests practical potential for real-time sentiment-aware forecasting and motivates further research into emotion-aware market analysis and real-time NLP signals in finance.

Abstract

This paper explores the intersection of Natural Language Processing (NLP) and financial analysis, focusing on the impact of sentiment analysis in stock price prediction. We employ BERTopic, an advanced NLP technique, to analyze the sentiment of topics derived from stock market comments. Our methodology integrates this sentiment analysis with various deep learning models, renowned for their effectiveness in time series and stock prediction tasks. Through comprehensive experiments, we demonstrate that incorporating topic sentiment notably enhances the performance of these models. The results indicate that topics in stock market comments provide implicit, valuable insights into stock market volatility and price trends. This study contributes to the field by showcasing the potential of NLP in enriching financial analysis and opens up avenues for further research into real-time sentiment analysis and the exploration of emotional and contextual aspects of market sentiment. The integration of advanced NLP techniques like BERTopic with traditional financial analysis methods marks a step forward in developing more sophisticated tools for understanding and predicting market behaviors.

BERTopic-Driven Stock Market Predictions: Unraveling Sentiment Insights

TL;DR

This work tackles stock price prediction by integrating investor sentiment extracted from online stock discussions with deep learning models. It introduces BERTopic as a topic-modeling backbone to derive topic-level sentiment from stock-related text and evaluates its impact when fused with LSTM, CNN, GAN, and CNN-LSTM architectures. Across experiments using BERT- and VADER-based sentiment signals, topic-enhanced predictions consistently improve metrics such as RMSE, MAE, and R2, demonstrating that topic-level insights contain valuable information about volatility and price trends. The study suggests practical potential for real-time sentiment-aware forecasting and motivates further research into emotion-aware market analysis and real-time NLP signals in finance.

Abstract

This paper explores the intersection of Natural Language Processing (NLP) and financial analysis, focusing on the impact of sentiment analysis in stock price prediction. We employ BERTopic, an advanced NLP technique, to analyze the sentiment of topics derived from stock market comments. Our methodology integrates this sentiment analysis with various deep learning models, renowned for their effectiveness in time series and stock prediction tasks. Through comprehensive experiments, we demonstrate that incorporating topic sentiment notably enhances the performance of these models. The results indicate that topics in stock market comments provide implicit, valuable insights into stock market volatility and price trends. This study contributes to the field by showcasing the potential of NLP in enriching financial analysis and opens up avenues for further research into real-time sentiment analysis and the exploration of emotional and contextual aspects of market sentiment. The integration of advanced NLP techniques like BERTopic with traditional financial analysis methods marks a step forward in developing more sophisticated tools for understanding and predicting market behaviors.
Paper Structure (30 sections, 12 equations, 13 figures, 3 tables)

This paper contains 30 sections, 12 equations, 13 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. The application of sentiment analysis in financial field
  4. Sentiment analysis to predict stock trend
  5. Development of text analysis technology based on topic model
  6. Application status and effect evaluation of BERTopic
  7. Methodology
  8. Data Collection
  9. Sentiment Model
  10. Valence Aware Dictionary and Sentiment Reasoner (VADER)
  11. BERTopic
  12. Sentence Embeddings
  13. Dimensionality Reduction
  14. Clustering
  15. Topic Generation
  16. ...and 15 more sections

Figures (13)

  • Figure 1: The usage of CountVectorizer
  • Figure 2: The structure of BERTopic
  • Figure 3: Long Short-Term Memory network.
  • Figure 4: Forget Gate.
  • Figure 5: Input Gate.
  • ...and 8 more figures