Building Resilient Information Ecosystems: Large LLM-Generated Dataset of Persuasion Attacks

TL;DR

This work presents a large, multi-model dataset of LLM-generated persuasion attacks (134,136 instances) covering 23 SemEval-2023 Task 3 techniques applied to 972 agency press releases from 10 agencies, generated by GPT-4, Gemma 2, and Llama 3.1 across two media forms. It combines domain and topic labeling with a Moral Foundations Theory analysis to reveal how attacks resonate with Care, Authority, and Loyalty, and compares model-specific tendencies in moral framing. The findings show distinct patterning across models (e.g., Gemma 2's strong Care/Authority resonance, Llama 3.1's Loyalty emphasis, GPT-4's Care-centric strategy) and highlight the need for proactive defenses and resilient communications—the concept of a reputation armor to withstand AI-driven persuasion. The dataset and analyses offer a practical resource for agencies to anticipate, detect, and counter AI-generated persuasion, with broad implications for information ecosystems and public trust.

Abstract

Organization's communication is essential for public trust, but the rise of generative AI models has introduced significant challenges by generating persuasive content that can form competing narratives with official messages from government and commercial organizations at speed and scale. This has left agencies in a reactive position, often unaware of how these models construct their persuasive strategies, making it more difficult to sustain communication effectiveness. In this paper, we introduce a large LLM-generated persuasion attack dataset, which includes 134,136 attacks generated by GPT-4, Gemma 2, and Llama 3.1 on agency news. These attacks span 23 persuasive techniques from SemEval 2023 Task 3, directed toward 972 press releases from ten agencies. The generated attacks come in two mediums, press release statements and social media posts, covering both long-form and short-form communication strategies. We analyzed the moral resonance of these persuasion attacks to understand their attack vectors. GPT-4's attacks mainly focus on Care, with Authority and Loyalty also playing a role. Gemma 2 emphasizes Care and Authority, while Llama 3.1 centers on Loyalty and Care. Analyzing LLM-generated persuasive attacks across models will enable proactive defense, allow to create the reputation armor for organizations, and propel the development of both effective and resilient communications in the information ecosystem.