The Wall Street Neophyte: A Zero-Shot Analysis of ChatGPT Over MultiModal Stock Movement Prediction Challenges

TL;DR

This study assesses whether ChatGPT can predict stock price movements in a zero-shot setting using multimodal inputs (historical price features and tweets). It systematically compares vanilla zero-shot and Chain-of-Thought prompted variants across three datasets (BIGDATA22, ACL18, CIKM18) against traditional baselines, using ACC and MCC as evaluation metrics. The results show that ChatGPT generally underperforms state-of-the-art methods, though tweet information provides consistent gains and CoT prompts offer better explanations, with performance being dataset-dependent. The work highlights the challenges of fusing multimodal financial signals in zero-shot LLMs and suggests that task-specific fine-tuning or training is needed to leverage LLMs effectively for finance applications.

Abstract

Recently, large language models (LLMs) like ChatGPT have demonstrated remarkable performance across a variety of natural language processing tasks. However, their effectiveness in the financial domain, specifically in predicting stock market movements, remains to be explored. In this paper, we conduct an extensive zero-shot analysis of ChatGPT's capabilities in multimodal stock movement prediction, on three tweets and historical stock price datasets. Our findings indicate that ChatGPT is a "Wall Street Neophyte" with limited success in predicting stock movements, as it underperforms not only state-of-the-art methods but also traditional methods like linear regression using price features. Despite the potential of Chain-of-Thought prompting strategies and the inclusion of tweets, ChatGPT's performance remains subpar. Furthermore, we observe limitations in its explainability and stability, suggesting the need for more specialized training or fine-tuning. This research provides insights into ChatGPT's capabilities and serves as a foundation for future work aimed at improving financial market analysis and prediction by leveraging social media sentiment and historical stock data.