Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLM-Based Adversarial Persuasion Attacks on Fact-Checking Systems

João A. Leite, Olesya Razuvayevskaya, Kalina Bontcheva, Carolina Scarton

TL;DR

The paper introduces persuasion injection attacks as a novel adversarial framework against automated fact-checking systems, leveraging generative LLMs to rewrite claims with persuasion techniques while preserving factual content. By decoupling verification from evidence retrieval, the authors show that persuasion can catastrophically degrade both components of AFC pipelines on FEVER and FEVEROUS, with oracle-guided attackers capable of driving claim-only accuracy to near zero even when gold evidence is provided. The study identifies especially harmful techniques, notably Manipulative Wording, Obfuscation, and rhetorical appeals, and demonstrates that these attacks can misalign claims and evidence, thereby compromising retrieval as well as reasoning. The work highlights the need for more robust AFC systems—potential remedies include stronger grounding, structured evidence, and defense mechanisms against rhetorical manipulation—and contributes open-source resources to support defensive research.

Abstract

Automated fact-checking (AFC) systems are susceptible to adversarial attacks, enabling false claims to evade detection. Existing adversarial frameworks typically rely on injecting noise or altering semantics, yet no existing framework exploits the adversarial potential of persuasion techniques, which are widely used in disinformation campaigns to manipulate audiences. In this paper, we introduce a novel class of persuasive adversarial attacks on AFCs by employing a generative LLM to rephrase claims using persuasion techniques. Considering 15 techniques grouped into 6 categories, we study the effects of persuasion on both claim verification and evidence retrieval using a decoupled evaluation strategy. Experiments on the FEVER and FEVEROUS benchmarks show that persuasion attacks can substantially degrade both verification performance and evidence retrieval. Our analysis identifies persuasion techniques as a potent class of adversarial attacks, highlighting the need for more robust AFC systems.

LLM-Based Adversarial Persuasion Attacks on Fact-Checking Systems

TL;DR

The paper introduces persuasion injection attacks as a novel adversarial framework against automated fact-checking systems, leveraging generative LLMs to rewrite claims with persuasion techniques while preserving factual content. By decoupling verification from evidence retrieval, the authors show that persuasion can catastrophically degrade both components of AFC pipelines on FEVER and FEVEROUS, with oracle-guided attackers capable of driving claim-only accuracy to near zero even when gold evidence is provided. The study identifies especially harmful techniques, notably Manipulative Wording, Obfuscation, and rhetorical appeals, and demonstrates that these attacks can misalign claims and evidence, thereby compromising retrieval as well as reasoning. The work highlights the need for more robust AFC systems—potential remedies include stronger grounding, structured evidence, and defense mechanisms against rhetorical manipulation—and contributes open-source resources to support defensive research.

Abstract

Automated fact-checking (AFC) systems are susceptible to adversarial attacks, enabling false claims to evade detection. Existing adversarial frameworks typically rely on injecting noise or altering semantics, yet no existing framework exploits the adversarial potential of persuasion techniques, which are widely used in disinformation campaigns to manipulate audiences. In this paper, we introduce a novel class of persuasive adversarial attacks on AFCs by employing a generative LLM to rephrase claims using persuasion techniques. Considering 15 techniques grouped into 6 categories, we study the effects of persuasion on both claim verification and evidence retrieval using a decoupled evaluation strategy. Experiments on the FEVER and FEVEROUS benchmarks show that persuasion attacks can substantially degrade both verification performance and evidence retrieval. Our analysis identifies persuasion techniques as a potent class of adversarial attacks, highlighting the need for more robust AFC systems.
Paper Structure (45 sections, 1 equation, 5 figures, 9 tables)

This paper contains 45 sections, 1 equation, 5 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Adversarial Attacks on AFC Systems
  4. Persuasion in NLP
  5. Methodology
  6. Persuasion Techniques.
  7. Variant Generation.
  8. Label-Invariance Validation.
  9. Attacker Capabilities.
  10. Experimental Setup
  11. Datasets
  12. AFC Classifiers
  13. Claim-Only Model (No Evidence).
  14. Gold-Evidence Model.
  15. Dev Set Performance.
  16. ...and 30 more sections

Figures (5)

  • Figure 1: Overview of a persuasion injection attack. An automated fact-checking (AFC) system correctly classifies the claim as False (left). Using our novel attack method (right), a generative LLM injects persuasion techniques into the claim; the same AFC system now incorrectly predicts True.
  • Figure 2: BM25 retrieval performance. Lower scores indicate more effective attacks.
  • Figure 3: Retrieval vs. classification vulnerabilities. Scores are averaged between FEVER and FEVEROUS. Evasion ASR computed under access to gold-evidence. Higher scores indicate more effective attacks.
  • Figure 4: Prompt template for persuasion injection attacks.
  • Figure 5: Prompt template for the paraphrase attack.