Towards Hybrid Intelligence in Journalism: Findings and Lessons Learnt from a Collaborative Analysis of Greek Political Rhetoric by ChatGPT and Humans

TL;DR

This experimental study investigates the capabilities of large language model (LLMs), and in particular OpenAI's ChatGPT, for analyzing political speech, evaluates its strengths and weaknesses, and highlights the essential role of human oversight in using AI in journalism projects and potentially other societal sectors.

Abstract

This chapter introduces a research project titled "Analyzing the Political Discourse: A Collaboration Between Humans and Artificial Intelligence", which was initiated in preparation for Greece's 2023 general elections. The project focused on the analysis of political leaders' campaign speeches, employing Artificial Intelligence (AI), in conjunction with an interdisciplinary team comprising journalists, a political scientist, and data scientists. The chapter delves into various aspects of political discourse analysis, including sentiment analysis, polarization, populism, topic detection, and Named Entities Recognition (NER). This experimental study investigates the capabilities of large language model (LLMs), and in particular OpenAI's ChatGPT, for analyzing political speech, evaluates its strengths and weaknesses, and highlights the essential role of human oversight in using AI in journalism projects and potentially other societal sectors. The project stands as an innovative example of human-AI collaboration (known also as "hybrid intelligence") within the realm of digital humanities, offering valuable insights for future initiatives.

Figures (5)

• Figure 1: Methodology of the hybrid intelligence political discourse analysis project.
• Figure 2: Treemap chart displaying the topics covered in the speeches of K. Mitsotakis (only for the parts that referred to their “political agenda”) during the first election period. Each box corresponds to a different topic, and its size is proportional to the percentage of paragraphs referring to this topic.
• Figure 3: Speedometer charts displaying the average sentiment for each political leader during the first (light gray) and second (dark gray) election periods. Red / gray / green areas correspond to negative / neutral / positive sentiments, respectively.
• Figure 4: Line chart depicting the sentiment evolution over the course of the second election period (x-axis) for K. Mitsotakis (dark blue line) and A. Tsipras (light blue line). The y-axis corresponds to sentiment values, from -1 to 1, and colored areas to negative (red), neutral (gray) and positive (green) sentiments.
• Figure 5: Step chart depicting the evolution of the polarization during a speech of A. Tsipras. The y-axis corresponds to polarization values, from 0 to 1, and colored areas to None/Low (gray), Medium (yellow) and High (red) levels of polarization sentiments.