On the Influence of Discourse Relations in Persuasive Texts

TL;DR

This work addresses how persuasion techniques map to discourse relations in persuasive text. It develops LLM-based DR classifiers to label PDTB level-2 senses and applies ensembles to annotate the SemEval 2023 PT dataset, creating five silver PT-DR corpora. Statistical analysis reveals six DR senses with robust positive associations to various PTs, notably showing strong links between Loaded Language/Exaggeration and Cause, and between Repetition and Purpose/Contrast. The findings advance automatic persuasion technique detection and offer insights for detecting online propaganda, with potential extensions to multi-label DR labeling and cross-language analysis.

Abstract

This paper investigates the relationship between Persuasion Techniques (PTs) and Discourse Relations (DRs) by leveraging Large Language Models (LLMs) and prompt engineering. Since no dataset annotated with both PTs and DRs exists, we took the SemEval 2023 Task 3 dataset labelled with 19 PTs as a starting point and developed LLM-based classifiers to label each instance of the dataset with one of the 22 PDTB 3.0 level-2 DRs. In total, four LLMs were evaluated using 10 different prompts, resulting in 40 unique DR classifiers. Ensemble models using different majority-pooling strategies were used to create 5 silver datasets of instances labelled with both persuasion techniques and level-2 PDTB senses. The silver dataset sizes vary from 1,281 instances to 204 instances, depending on the majority pooling technique used. Statistical analysis of these silver datasets shows that six discourse relations (namely Cause, Purpose, Contrast, Cause+Belief, Concession, and Condition) play a crucial role in persuasive texts, especially in the use of Loaded Language, Exaggeration/Minimisation, Repetition and to cast Doubt. This insight can contribute to detecting online propaganda and misinformation, as well as to our general understanding of effective communication.

Figures (7)

• Figure 1: Overall Methodology
• Figure 2: Excerpt from Prompt 3 showing four PDTB level 2 senses with definitions and examples. [...] refers to the description of the remaining PDTB level 2 senses.
• Figure 3: Impact of Prompts on LLM Performance for DR Classification. Each bar represents the average of two independent runs per LLM on that prompt. Prompt average shows the mean performance across all four LLMs for each prompt.
• Figure 4: Macro F1 Score of the Ensemble DR Classifiers
• Figure 5: Example of a Dually Annotated Instance in a Silver Dataset.