Using Persuasive Writing Strategies to Explain and Detect Health Misinformation

TL;DR

This work tackles health misinformation by proposing a hierarchical persuasive writing strategies framework and a matching annotated dataset. It demonstrates that incorporating persuasive strategies as intermediate labels can improve misinformation detection and provide explanations, leveraging both RoBERTa-based models and large language models like GPT-4 through fine-tuning and prompting. The authors construct a health-focused subset from MultiFC, compile full articles, and develop a three-level annotation codebook with high inter-annotator reliability. They show that ground-truth strategy labels boost accuracy and that even predicted strategies can aid detection in prompting-based approaches, offering a valuable resource for future research and media literacy tools. Collectively, the study advances both methodology and resources for interpretable health misinformation detection at scale.

Abstract

Nowadays, the spread of misinformation is a prominent problem in society. Our research focuses on aiding the automatic identification of misinformation by analyzing the persuasive strategies employed in textual documents. We introduce a novel annotation scheme encompassing common persuasive writing tactics to achieve our objective. Additionally, we provide a dataset on health misinformation, thoroughly annotated by experts utilizing our proposed scheme. Our contribution includes proposing a new task of annotating pieces of text with their persuasive writing strategy types. We evaluate fine-tuning and prompt-engineering techniques with pre-trained language models of the BERT family and the generative large language models of the GPT family using persuasive strategies as an additional source of information. We evaluate the effects of employing persuasive strategies as intermediate labels in the context of misinformation detection. Our results show that those strategies enhance accuracy and improve the explainability of misinformation detection models. The persuasive strategies can serve as valuable insights and explanations, enabling other models or even humans to make more informed decisions regarding the trustworthiness of the information.

Figures (8)

• Figure 1: Persuasive Strategies annotation hierarchy scheme.
• Figure 2: Persuasive strategy labeling example.
• Figure 3: An example of misinformation detection prompt for GPT model in a zero-shot setting. The prompt structure would vary based on the input sources. More prompt templates can be found in Appendix \ref{['appendix:gpt-experiments']}.
• Figure 4: The prompt structure for the explained misinformation detection using persuasive strategies with an example response.
• Figure 5: An example of persuasive strategy detection using GPT-3.5.