Human Interest or Conflict? Leveraging LLMs for Automated Framing Analysis in TV Shows

TL;DR

This paper proposes a novel approach to use prompt-engineering to identify the framing of spoken content in television programs, and indicates that prompt-engineering LLMs can be used as a support tool to identify frames.

Abstract

In the current media landscape, understanding the framing of information is crucial for critical consumption and informed decision making. Framing analysis is a valuable tool for identifying the underlying perspectives used to present information, and has been applied to a variety of media formats, including television programs. However, manual analysis of framing can be time-consuming and labor-intensive. This is where large language models (LLMs) can play a key role. In this paper, we propose a novel approach to use prompt-engineering to identify the framing of spoken content in television programs. Our findings indicate that prompt-engineering LLMs can be used as a support tool to identify frames, with agreement rates between human and machine reaching up to 43\%. As LLMs are still under development, we believe that our approach has the potential to be refined and further improved. The potential of this technology for interactive media applications is vast, including the development of support tools for journalists, educational resources for students of journalism learning about framing and related concepts, and interactive media experiences for audiences.

Figures (8)

• Figure 1: Example of the structure (text to annotate + definition of frames + questions ) of the form to do the annotation.
• Figure 2: Example of the prompt used for frame classification, given a transcript.
• Figure 3: Agreement between human annotator and GPT-3.5 on classification of EenVandaag transcripts into five categories: Conflict, Economic, Human Interest, Morality and Responsibility.
• Figure 4: Agreement between human annotator and GPT-3.5 on classification of Niewwsuur transcripts into five categories: Conflict, Economic, Human Interest, Morality and Responsibility.
• Figure 5: Relative frequencies of agreement and disagreement between human and GPT-3.5 annotations for EenVandaag/Nieuwsuur transcripts across text length bins.