Robust Fake News Detection using Large Language Models under Adversarial Sentiment Attacks

TL;DR

The paper tackles the vulnerability of fake news detectors to sentiment-based adversarial manipulation by LLMs. It introduces AdSent, a two-part framework combining sentiment-based attacks and sentiment-agnostic training to maintain consistent veracity predictions across sentiment-altered articles. Extensive experiments across PolitiFact, GossipCop, and LUN show that standard detectors suffer substantial performance drops under sentiment manipulation, while AdSent achieves superior robustness and generalization to unseen data and attack types. The results highlight the practical value of sentiment-agnostic defenses for reliable misinformation detection and point to future work in multimodal settings and broader news values.

Abstract

Misinformation and fake news have become a pressing societal challenge, driving the need for reliable automated detection methods. Prior research has highlighted sentiment as an important signal in fake news detection, either by analyzing which sentiments are associated with fake news or by using sentiment and emotion features for classification. However, this poses a vulnerability since adversaries can manipulate sentiment to evade detectors especially with the advent of large language models (LLMs). A few studies have explored adversarial samples generated by LLMs, but they mainly focus on stylistic features such as writing style of news publishers. Thus, the crucial vulnerability of sentiment manipulation remains largely unexplored. In this paper, we investigate the robustness of state-of-the-art fake news detectors under sentiment manipulation. We introduce AdSent, a sentiment-robust detection framework designed to ensure consistent veracity predictions across both original and sentiment-altered news articles. Specifically, we (1) propose controlled sentiment-based adversarial attacks using LLMs, (2) analyze the impact of sentiment shifts on detection performance. We show that changing the sentiment heavily impacts the performance of fake news detection models, indicating biases towards neutral articles being real, while non-neutral articles are often classified as fake content. (3) We introduce a novel sentiment-agnostic training strategy that enhances robustness against such perturbations. Extensive experiments on three benchmark datasets demonstrate that AdSent significantly outperforms competitive baselines in both accuracy and robustness, while also generalizing effectively to unseen datasets and adversarial scenarios.

Figures (5)

• Figure 1: Example of a sentiment-based adversarial attack, where a fake news instance (red box) is manipulated to convey a positive tone using LLM, resulting in a misclassification by the detector.
• Figure 2: Overview of the proposed AdSent framework for sentiment-robust fake news detection. The framework consists of two main components: an attack module (red box) and a defense module (green box). Given an input document $D_i$, the model predicts whether it is fake or real.
• Figure 3: Fact-preservation accuracy (%) across different annotated variants of the PolitiFact test set, evaluated by humans
• Figure 4: Macro-F1 (F1) (%) and percentage of flips in predictions for LLaMA-3.1-8b-Instruct model on different variants of Politifact test set.
• Figure 5: Qualitative comparison of three models for adversarial sentiment analysis on Politifact dataset. Green text box indicates correct predictions; red text boxes indicates incorrect ones. red borders show fake news document.