LLM Generated Distribution-Based Prediction of US Electoral Results, Part I

TL;DR

The use of distribution-based prediction is demonstrated in the context of recent United States presidential election, showing that this method can be used to determine task specific bias, prompt noise, and algorithmic fidelity.

Abstract

This paper introduces distribution-based prediction, a novel approach to using Large Language Models (LLMs) as predictive tools by interpreting output token probabilities as distributions representing the models' learned representation of the world. This distribution-based nature offers an alternative perspective for analyzing algorithmic fidelity, complementing the approach used in silicon sampling. We demonstrate the use of distribution-based prediction in the context of recent United States presidential election, showing that this method can be used to determine task specific bias, prompt noise, and algorithmic fidelity. This approach has significant implications for assessing the reliability and increasing transparency of LLM-based predictions across various domains.

Figures (10)

• Figure 1: Model predictions for Iowa in the 2020 presidential election, showing a high concentration of probability mass around the actual vote percentage.
• Figure 2: Model-predicted electoral college outcome for the 2020 election, matching the actual results.
• Figure 3: Actual electoral college outcome for the 2020 election.
• Figure 4: Model predictions for Iowa (typically a Republican leaning state) in the 2024 presidential election, showing a distribution for the model's prediction that is distinct from its prediction for the 2020 election.
• Figure 5: Model predictions for California (a strongly Democratic state)