Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

When Agents Persuade: Propaganda Generation and Mitigation in LLMs

Julia Jose, Ritik Roongta, Rachel Greenstadt

TL;DR

It is found that fine-tuning LLM-based agents with propaganda objectives significantly reduces their tendency to generate such content, with ORPO (Odds Ratio Preference Optimization) proving most effective.

Abstract

Despite their wide-ranging benefits, LLM-based agents deployed in open environments can be exploited to produce manipulative material. In this study, we task LLMs with propaganda objectives and analyze their outputs using two domain-specific models: one that classifies text as propaganda or non-propaganda, and another that detects rhetorical techniques of propaganda (e.g., loaded language, appeals to fear, flag-waving, name-calling). Our findings show that, when prompted, LLMs exhibit propagandistic behaviors and use a variety of rhetorical techniques in doing so. We also explore mitigation via Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and ORPO (Odds Ratio Preference Optimization). We find that fine-tuning significantly reduces their tendency to generate such content, with ORPO proving most effective.

When Agents Persuade: Propaganda Generation and Mitigation in LLMs

TL;DR

It is found that fine-tuning LLM-based agents with propaganda objectives significantly reduces their tendency to generate such content, with ORPO (Odds Ratio Preference Optimization) proving most effective.

Abstract

Despite their wide-ranging benefits, LLM-based agents deployed in open environments can be exploited to produce manipulative material. In this study, we task LLMs with propaganda objectives and analyze their outputs using two domain-specific models: one that classifies text as propaganda or non-propaganda, and another that detects rhetorical techniques of propaganda (e.g., loaded language, appeals to fear, flag-waving, name-calling). Our findings show that, when prompted, LLMs exhibit propagandistic behaviors and use a variety of rhetorical techniques in doing so. We also explore mitigation via Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and ORPO (Odds Ratio Preference Optimization). We find that fine-tuning significantly reduces their tendency to generate such content, with ORPO proving most effective.
Paper Structure (34 sections, 2 figures, 13 tables)

This paper contains 34 sections, 2 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Generative AI, Disinformation, and Propaganda
  4. Propaganda Detection
  5. Methodology
  6. Data and Model Training
  7. Propaganda Detection Model
  8. Rhetorical Techniques Detection Model
  9. LLM Propaganda Generation
  10. Human Validation of LLM-generated Propaganda
  11. Supervised Fine-Tuning and Preference Alignment
  12. Results
  13. RQ1: Can LLMs generate propaganda?
  14. Propaganda Classification Rates
  15. Error Analysis
  16. ...and 19 more sections

Figures (2)

  • Figure 1: Clustered heatmap showing the average number of occurrences of six rhetorical techniques across human-written and LLM-generated articles. Columns represent each dataset (propaganda vs. non-propaganda for humans, GPT-4o, Llama-3.1, and Mistral Small 3), and darker shades indicate more frequent use of a technique.
  • Figure 2: (Left) Frequency of techniques across fine-tuning methods. (Right) Rhetorical techniques across fine-tuned and unfine-tuned versions.