Speaker attribution in German parliamentary debates with QLoRA-adapted large language models

TL;DR

The potential of the large language model family Llama 2 to automate speaker attribution in German parliamentary debates from 2017-2021 is studied and a promising avenue for computational analysis of political discourse and the development of semantic role labeling systems is revealed.

Abstract

The growing body of political texts opens up new opportunities for rich insights into political dynamics and ideologies but also increases the workload for manual analysis. Automated speaker attribution, which detects who said what to whom in a speech event and is closely related to semantic role labeling, is an important processing step for computational text analysis. We study the potential of the large language model family Llama 2 to automate speaker attribution in German parliamentary debates from 2017-2021. We fine-tune Llama 2 with QLoRA, an efficient training strategy, and observe our approach to achieve competitive performance in the GermEval 2023 Shared Task On Speaker Attribution in German News Articles and Parliamentary Debates. Our results shed light on the capabilities of large language models in automating speaker attribution, revealing a promising avenue for computational analysis of political discourse and the development of semantic role labeling systems.

Figures (3)

• Figure 1: Sentence from the Train dataset with three annotations. The sentence was split into three samples by spaCy (splitting points are indicated by ‡). This segmentation also occurs at not-punctuated positions, as seen in the example sentence ("… rassistischen Positionen ‡ haben die Rechtsradikalen …"). This behavior is due to the data provided by "Open Bundestag", where comments from other members of parliament during an otherwise coherent paragraph force this unintuitive segmentation into two separate paragraphs GermEval2023. As seen in Annotation 2, there can be annotations consisting of only cue word(s). Annotation 1 and Annotation 3 show that annotated roles can span multiple samples.
• Figure 2: Example cue prompt and desired model response for the sample "denn wir wissen: Neben ihren rassistischen Positionen" with the cues "wissen" and "Positionen". Shaded in gray are the parts of the prompt and response that are sample dependent. The prompt is used as the Input sequence for training and inference, while the Output sequence contains the desired response with the cues. The end-of-sentence token "</s>" is used to indicate the end of the Output sequence.
• Figure 3: Example role prompt and desired model response for the sample "denn wir wissen: Neben ihren rassistischen Positionen" with the cue "wissen". Since roles can be contained in samples different from the one containing the cue, we concatenated the sample with the next two samples of the same speech (transitions between samples are indicated by ‡). Shaded in gray are the parts of the prompt and response that are sample dependent. Similar to the cue prompt, the role prompt is used as the Input sequence for training and inference, while the Output sequence contains the desired response. We append the end-of-sentence token "</s>" to the Output.