Can LLMs Learn Macroeconomic Narratives from Social Media?

TL;DR

This work probes whether macroeconomic narratives extracted from social media can enhance short-term macro forecasts. It combines two Twitter datasets with three macro indicators ($FFR$, $S&P 500$, $VIX$) and compares raw-tweet representations, embeddings, and explicit LLM-derived narrative analyses in prediction tasks across multiple horizons. Across extensive experiments, narrative signals yield only marginal improvements over financial baselines, suggesting that current NLP/LMM approaches struggle to translate public narratives into robust macro predictors. The study provides NLP tools and a framework for testing Narrative Economics at the macro level, highlighting the need for new models and tasks to more accurately capture the economy-wide impact of shared narratives.

Abstract

This study empirically tests the $\textit{Narrative Economics}$ hypothesis, which posits that narratives (ideas that are spread virally and affect public beliefs) can influence economic fluctuations. We introduce two curated datasets containing posts from X (formerly Twitter) which capture economy-related narratives (Data will be shared upon paper acceptance). Employing Natural Language Processing (NLP) methods, we extract and summarize narratives from the tweets. We test their predictive power for $\textit{macroeconomic}$ forecasting by incorporating the tweets' or the extracted narratives' representations in downstream financial prediction tasks. Our work highlights the challenges in improving macroeconomic models with narrative data, paving the way for the research community to realistically address this important challenge. From a scientific perspective, our investigation offers valuable insights and NLP tools for narrative extraction and summarization using Large Language Models (LLMs), contributing to future research on the role of narratives in economics.

Figures (12)

• Figure 1: Textual models' pipelines to represent textual data as part of the prediction pipeline. (a) Daily sentiments. (b) Individual and joint tweets LLM's representations. (c) LLM analysis for prediction and as input to a subsequent prediction model.
• Figure 2: Temporal distribution of top three narratives from the "economics" dataset, extracted by RELATIO ash2021relatio (see \ref{['sec:existing_efforts']}): UK's Brexit, Greece's financial debt, and Russia's financial crises. We can see the evolving nature of these narratives over time, where the distribution is aligned with real-life related events.
• Figure 3: Snippet of LLM-based analysis for 29/08/2022 to 28/09/2022. In this time period the Federal Reserve raised the interest rates in an effort to combat inflation, the US Supreme Court ruled that the Biden administration could not extend the pause on student loan payments, and more. See full analysis in App. \ref{['subapp:llm_analysis']}
• Figure 4: VIX "next value" prediction for 1/7-days horizons. The F and TF baselines outperform all models in 1- and 7-day horizons, respectively, suggesting all models struggle to learn from tweets for the prediction.
• Figure 5: Blocking split of FFR data, isolating time-periods with consistent distributions.