PreBit -- A multimodal model with Twitter FinBERT embeddings for extreme price movement prediction of Bitcoin

TL;DR

Bitcoin's extreme price volatility poses predictive challenges; this work introduces PreBit, a multimodal model that fuses FinBERT-based tweet embeddings with traditional TA data. Tweets are embedded with 768-dimensional FinBERT vectors and fed to CNNs (parallel and sequential) while candlestick and indicator data drive an SVM TA model, with a fusion stage combining both predictions. The authors release the PreBit dataset (9.4M tweets across 2015–2021) and demonstrate through ablations that incorporating Twitter content improves upward-movement prediction, and that a 0.95 threshold fusion strategy achieves favorable risk-adjusted backtest metrics compared to baselines. Overall, PreBit offers a practical framework and public resource for multimodal cryptocurrency prediction and risk-aware trading strategies, while noting limitations related to data representativeness and non-stationarity.

Abstract

Bitcoin, with its ever-growing popularity, has demonstrated extreme price volatility since its origin. This volatility, together with its decentralised nature, make Bitcoin highly subjective to speculative trading as compared to more traditional assets. In this paper, we propose a multimodal model for predicting extreme price fluctuations. This model takes as input a variety of correlated assets, technical indicators, as well as Twitter content. In an in-depth study, we explore whether social media discussions from the general public on Bitcoin have predictive power for extreme price movements. A dataset of 5,000 tweets per day containing the keyword `Bitcoin' was collected from 2015 to 2021. This dataset, called PreBit, is made available online. In our hybrid model, we use sentence-level FinBERT embeddings, pretrained on financial lexicons, so as to capture the full contents of the tweets and feed it to the model in an understandable way. By combining these embeddings with a Convolutional Neural Network, we built a predictive model for significant market movements. The final multimodal ensemble model includes this NLP model together with a model based on candlestick data, technical indicators and correlated asset prices. In an ablation study, we explore the contribution of the individual modalities. Finally, we propose and backtest a trading strategy based on the predictions of our models with varying prediction threshold and show that it can used to build a profitable trading strategy with a reduced risk over a `hold' or moving average strategy.

Figures (11)

• Figure 1: Schematic overview of the paper.
• Figure 2: Top 20 most frequent words in the tweets in our dataset.
• Figure 3: Visualisation of selected TA features.
• Figure 4: Correlation matrix for the TA indicators.
• Figure 5: Proposed multimodal hybrid model.