The Facade of Truth: Uncovering and Mitigating LLM Susceptibility to Deceptive Evidence

TL;DR

This paper exposes a fundamental vulnerability in LLMs: sophisticated, hard-to-falsify evidence can destabilize internal beliefs despite resistance to overt misinformation. It presents MisBelief, a multi-agent framework that incrementally generates deceptive evidence through Planner, Reviewer, and Refiner roles, producing 4,800 instances across eight domains and three difficulty levels to probe robustness across seven LLMs. Results show substantial belief shifts (average +93.0%) and downstream risk in recommendations, with reasoning-optimized models especially susceptible. To counter this, the authors propose Deceptive Intent Shielding (DIS), which uses an Analyst to infer deceptive intent behind evidence and issue warnings, consistently mitigating belief shifts and improving downstream reliability. The work highlights the need for intent-aware alignment alongside fact-based defenses, and demonstrates DIS’s feasibility and usefulness for human users in reducing the impact of deceptive evidence.

Abstract

To reliably assist human decision-making, LLMs must maintain factual internal beliefs against misleading injections. While current models resist explicit misinformation, we uncover a fundamental vulnerability to sophisticated, hard-to-falsify evidence. To systematically probe this weakness, we introduce MisBelief, a framework that generates misleading evidence via collaborative, multi-round interactions among multi-role LLMs. This process mimics subtle, defeasible reasoning and progressive refinement to create logically persuasive yet factually deceptive claims. Using MisBelief, we generate 4,800 instances across three difficulty levels to evaluate 7 representative LLMs. Results indicate that while models are robust to direct misinformation, they are highly sensitive to this refined evidence: belief scores in falsehoods increase by an average of 93.0\%, fundamentally compromising downstream recommendations. To address this, we propose Deceptive Intent Shielding (DIS), a governance mechanism that provides an early warning signal by inferring the deceptive intent behind evidence. Empirical results demonstrate that DIS consistently mitigates belief shifts and promotes more cautious evidence evaluation.

Figures (7)

• Figure 1: Illustrative example of belief manipulation via fabricated evidence. Conditioning on deceptive evidence increases the LLM’s belief score for the claim from 1/5 to 4/5, illustrating how hard-to-falsify evidence can induce unwarranted confidence, even when contradictory factual context is readily available.
• Figure 2: (a) MisBelief, an evaluation framework for assessing how misleading evidence influences LLM belief formation. Each instance consists of a misinformation claim with one of three detection difficulty levels, a set of deceptive supporting evidence iteratively generated through multi-role LLM interactions, and a belief score that measures the model’s response. (b) Deceptive Intent Shielding (DIS) strategy, which employs an analyst LLM to infer the intent underlying the evidence and issue early warnings, reducing unwarranted trust in misinformation.
• Figure 3: Internal validation of evidence credibility, where higher scores indicate greater perceived credibility. After two rounds of refinement, the generated evidence consistently attains high credibility scores.
• Figure 4: LLM belief scores increase with successive refinement rounds. The largest gain occurs during the first three rounds, after which marginal gains diminish.
• Figure 5: LLM belief scores under varying numbers of supporting evidence. Even a single evidence instance can substantially increase belief in misinformation.