Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Can LLMs Help Predict Elections? (Counter)Evidence from the World's Largest Democracy

Pratik Gujral, Kshitij Awaldhi, Navya Jain, Bhavuk Bhandula, Abhijnan Chakraborty

TL;DR

This work investigates whether zero-shot LLMs can forecast election outcomes in India using Twitter data, comparing their performance to traditional opinion and exit polls. By collecting and preserving a large tweet dataset from the 2022 Punjab and UP elections and applying Llama-2-13B and Zephyr-7B to extract party/state sentiment, the authors map sentiment signals to vote shares through multiple schemes and ensemble them via PoLLMster. The results show that PoLLMster predictions closely track actual outcomes, often outperforming polls, underscoring the potential of LLM-based social media analytics for political forecasting in multilingual, diverse democracies. The study also discusses notable limitations, such as sampling bias and ethical considerations, and outlines directions for future improvements with larger datasets and more advanced models.

Abstract

The study of how social media affects the formation of public opinion and its influence on political results has been a popular field of inquiry. However, current approaches frequently offer a limited comprehension of the complex political phenomena, yielding inconsistent outcomes. In this work, we introduce a new method: harnessing the capabilities of Large Language Models (LLMs) to examine social media data and forecast election outcomes. Our research diverges from traditional methodologies in two crucial respects. First, we utilize the sophisticated capabilities of foundational LLMs, which can comprehend the complex linguistic subtleties and contextual details present in social media data. Second, we focus on data from X (Twitter) in India to predict state assembly election outcomes. Our method entails sentiment analysis of election-related tweets through LLMs to forecast the actual election results, and we demonstrate the superiority of our LLM-based method against more traditional exit and opinion polls. Overall, our research offers valuable insights into the unique dynamics of Indian politics and the remarkable impact of social media in molding public attitudes within this context.

Can LLMs Help Predict Elections? (Counter)Evidence from the World's Largest Democracy

TL;DR

This work investigates whether zero-shot LLMs can forecast election outcomes in India using Twitter data, comparing their performance to traditional opinion and exit polls. By collecting and preserving a large tweet dataset from the 2022 Punjab and UP elections and applying Llama-2-13B and Zephyr-7B to extract party/state sentiment, the authors map sentiment signals to vote shares through multiple schemes and ensemble them via PoLLMster. The results show that PoLLMster predictions closely track actual outcomes, often outperforming polls, underscoring the potential of LLM-based social media analytics for political forecasting in multilingual, diverse democracies. The study also discusses notable limitations, such as sampling bias and ethical considerations, and outlines directions for future improvements with larger datasets and more advanced models.

Abstract

The study of how social media affects the formation of public opinion and its influence on political results has been a popular field of inquiry. However, current approaches frequently offer a limited comprehension of the complex political phenomena, yielding inconsistent outcomes. In this work, we introduce a new method: harnessing the capabilities of Large Language Models (LLMs) to examine social media data and forecast election outcomes. Our research diverges from traditional methodologies in two crucial respects. First, we utilize the sophisticated capabilities of foundational LLMs, which can comprehend the complex linguistic subtleties and contextual details present in social media data. Second, we focus on data from X (Twitter) in India to predict state assembly election outcomes. Our method entails sentiment analysis of election-related tweets through LLMs to forecast the actual election results, and we demonstrate the superiority of our LLM-based method against more traditional exit and opinion polls. Overall, our research offers valuable insights into the unique dynamics of Indian politics and the remarkable impact of social media in molding public attitudes within this context.
Paper Structure (19 sections, 2 equations, 4 figures, 2 tables)

This paper contains 19 sections, 2 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Background on the 2022 Punjab and UP Assembly Elections
  4. Methodology
  5. Dataset collection
  6. Tweets
  7. Opinion Polls and Exit Polls
  8. Data Preparation
  9. Tweets Dataset
  10. Opinion and Exit Polls
  11. Identify sentiment using LLM
  12. Post-processing
  13. Mapping sentiment scores to vote shares
  14. Experimental Evaluation
  15. Vote share mapping methods
  16. ...and 4 more sections

Figures (4)

  • Figure 1: Absolute error/deviation of the predicted vote shares using LLMs and opinion polls and exit polls from the actual vote shares
  • Figure 2: Predicted vote shares from using different mapping methods individually vs. PoLLMster (Zephyr7B-$\beta$) vs. the actual vote share for Punjab
  • Figure 3: Punjab - Exit and opinion polls from various pollsters
  • Figure 4: Uttar Pradesh - Exit and opinion polls from various pollsters