Proactive Defense: Compound AI for Detecting Persuasion Attacks and Measuring Inoculation Effectiveness

TL;DR

The paper addresses the challenge of detecting and mitigating targeted persuasion attacks in information ecosystems. It introduces BRIES, a compound AI framework with Detector, Defender, and Assessor agents, plus a Twister component for adversarial content, and relies on causal inference (SEM and ATE) to measure inoculation effectiveness. Key findings show GPT-4 excels at detecting complex persuasion techniques, while open-source models struggle with subtle rhetoric; prompt design and temperature significantly influence performance. The work offers actionable inoculation strategies, model- and technique-specific guidelines, and a foundation for proactive cognitive resilience in real-world information environments.

Abstract

This paper introduces BRIES, a novel compound AI architecture designed to detect and measure the effectiveness of persuasion attacks across information environments. We present a system with specialized agents: a Twister that generates adversarial content employing targeted persuasion tactics, a Detector that identifies attack types with configurable parameters, a Defender that creates resilient content through content inoculation, and an Assessor that employs causal inference to evaluate inoculation effectiveness. Experimenting with the SemEval 2023 Task 3 taxonomy across the synthetic persuasion dataset, we demonstrate significant variations in detection performance across language agents. Our comparative analysis reveals significant performance disparities with GPT-4 achieving superior detection accuracy on complex persuasion techniques, while open-source models like Llama3 and Mistral demonstrated notable weaknesses in identifying subtle rhetorical, suggesting that different architectures encode and process persuasive language patterns in fundamentally different ways. We show that prompt engineering dramatically affects detection efficacy, with temperature settings and confidence scoring producing model-specific variations; Gemma and GPT-4 perform optimally at lower temperatures while Llama3 and Mistral show improved capabilities at higher temperatures. Our causal analysis provides novel insights into socio-emotional-cognitive signatures of persuasion attacks, revealing that different attack types target specific cognitive dimensions. This research advances generative AI safety and cognitive security by quantifying LLM-specific vulnerabilities to persuasion attacks and delivers a framework for enhancing human cognitive resilience through structured interventions before exposure to harmful content.

Figures (4)

• Figure 1: Building Resilient Information Ecosystems framework for content inoculations to enable rapid experimentation and measurement of effectiveness for proactive defense.
• Figure 2: Persuasion attack detection results across 23 attack types for open (Gemma2, LLaMa3, Mistral and Phi3) vs. closed models (GPT4) with ($s_0$) = asking for a confidence score, (0) not asking.
• Figure 3: Persuasion attack detection results across 23 attack types for open vs. closed models with (0) models use the base prompt while ($d_0$) use a modified prompt that includes a description of each attack listed. Models have temperature set to 0.
• Figure 4: Persuasion attack detection results across 23 attack types for open vs. closed models (GPT4) with temperatures set up to (0) or (1).