Emotionally Charged, Logically Blurred: AI-driven Emotional Framing Impairs Human Fallacy Detection

TL;DR

This work investigates how AI-generated emotional framing interacts with logical fallacies and human judgments of argument strength. By leveraging eight LLMs to inject six emotions using four framing strategies while preserving argument structure, the authors build a 1,000-argument emotionally framed dataset and conduct a controlled human study. They find that AI-driven emotional framing reduces human fallacy-detection performance by about 14.5% in F1, while the rise in perceived convincingness is modest; perceived enjoyment enhances detection whereas fear and sadness impair it, and these emotions correlate with higher persuasiveness. The study highlights both the vulnerability of human reasoning to emotion-laden persuasion and the need for safeguards against AI-driven manipulation in argumentation.

Abstract

Logical fallacies are common in public communication and can mislead audiences; fallacious arguments may still appear convincing despite lacking soundness, because convincingness is inherently subjective. We present the first computational study of how emotional framing interacts with fallacies and convincingness, using large language models (LLMs) to systematically change emotional appeals in fallacious arguments. We benchmark eight LLMs on injecting emotional appeal into fallacious arguments while preserving their logical structures, then use the best models to generate stimuli for a human study. Our results show that LLM-driven emotional framing reduces human fallacy detection in F1 by 14.5% on average. Humans perform better in fallacy detection when perceiving enjoyment than fear or sadness, and these three emotions also correlate with significantly higher convincingness compared to neutral or other emotion states. Our work has implications for AI-driven emotional manipulation in the context of fallacious argumentation.

Figures (8)

• Figure 1: Label distributions: percentages of each label.
• Figure 2: Average convincingness scores (z-score normalized) for each emotion and fallacy category based on human annotations (perceived convincingness, emotions, and fallacies). Full pairwise t-test results appear in Tables \ref{['tab:ttest_conv_fallacy']} and \ref{['tab:ttest_conv_emo']} (Appendix \ref{['app:ttest']}).
• Figure 3: Accuracy of the prompt-based argument classifier from chen-eger-2025-emotions using different LLMs on our dataset in both argument classification and claim generation.
• Figure 4: Screenshots of Google Sheets used for annotations on reasoning preservation, coherence, and claim correctness.
• Figure 5: Screenshots of the Google Forms annotation interface showing guidelines for emotional appeal and emotion match evaluation in §\ref{['sec:frame']}.