UKElectionNarratives: A Dataset of Misleading Narratives Surrounding Recent UK General Elections

TL;DR

This work introduces UKElectionNarratives, the first human-annotated dataset of misleading narratives around UK General Elections (2019 and 2024) and a multi-level codebook capturing 10 super-narratives and 32 narratives across Europe. It combines literature-grounded taxonomy development, large-scale Twitter data collection, a three-stage human annotation pipeline, and a benchmarking study comparing RoBERTa-based PLMs with GPT-4o on narrative detection, including zero-shot and few-shot prompts with narrative descriptions. The authors demonstrate that GPT-4o substantially outperforms RoBERTa models and show that incorporating narrative descriptions into prompts improves performance, with the best results achieved via zero-shot prompts augmented by narrative descriptions. They also perform BERTopic-based topic modeling to link detected topics to narratives and analyze cross-election topic similarity, highlighting persistent themes such as EU relations, immigration, gender, antisemitism, and Islamophobia, and discuss potential use cases, limitations, and future directions for scalable narrative detection and automatic annotation in European electoral contexts.

Abstract

Misleading narratives play a crucial role in shaping public opinion during elections, as they can influence how voters perceive candidates and political parties. This entails the need to detect these narratives accurately. To address this, we introduce the first taxonomy of common misleading narratives that circulated during recent elections in Europe. Based on this taxonomy, we construct and analyse UKElectionNarratives: the first dataset of human-annotated misleading narratives which circulated during the UK General Elections in 2019 and 2024. We also benchmark Pre-trained and Large Language Models (focusing on GPT-4o), studying their effectiveness in detecting election-related misleading narratives. Finally, we discuss potential use cases and make recommendations for future research directions using the proposed codebook and dataset.

Figures (7)

• Figure 1: UKElectionNarratives construction approach. We first collect relevant tweets using a set of keywords, hashtags, and users. To select tweets with potential narratives, we select the highly retweeted tweets and those that 3 LLMs assigned identical narrative classes. The filtered tweets were then annotated by humans following a three-stage process.
• Figure 2: Annotation guidelines.
• Figure 3: Selecting the first choice during annotation.
• Figure 4: Selecting the second choice during annotation.
• Figure 5: A second choice was not required if the confidence score is above 3.