Vulnerability of LLMs' Belief Systems? LLMs Belief Resistance Check Through Strategic Persuasive Conversation Interventions

TL;DR

This study probes how LLM belief states resist or succumb to strategic persuasion using the SMCR framework across five models and three domains, employing a multi-turn belief-tracking setup. It reveals substantial, model- and domain-specific vulnerability, with small models showing near-complete compliance and medical QA being especially fragile. Contrary to human data, meta-cognition prompting often degrades robustness, while adversarial fine-tuning yields uneven gains across architectures, suggesting defenses must be interaction-aware rather than focused solely on content. The work highlights the need for robust, belief-stability-centric defenses and ethical considerations around manipulating model beliefs in real-world deployments.

Abstract

Large Language Models (LLMs) are increasingly employed in various question-answering tasks. However, recent studies showcase that LLMs are susceptible to persuasion and could adopt counterfactual beliefs. We present a systematic evaluation of LLM susceptibility to persuasion under the Source--Message--Channel--Receiver (SMCR) communication framework. Across five mainstream Large Language Models (LLMs) and three domains (factual knowledge, medical QA, and social bias), we analyze how different persuasive strategies influence belief stability over multiple interaction turns. We further examine whether meta-cognition prompting (i.e., eliciting self-reported confidence) affects resistance to persuasion. Results show that smaller models exhibit extreme compliance, with over 80% of belief changes occurring at the first persuasive turn (average end turn of 1.1--1.4). Contrary to expectations, meta-cognition prompting increases vulnerability by accelerating belief erosion rather than enhancing robustness. Finally, we evaluate adversarial fine-tuning as a defense. While GPT-4o-mini achieves near-complete robustness (98.6%) and Mistral~7B improves substantially (35.7% $\rightarrow$ 79.3%), Llama models remain highly susceptible (<14%) even when fine-tuned on their own failure cases. Together, these findings highlight substantial model-dependent limits of current robustness interventions and offer guidance for developing more trustworthy LLMs.

Figures (11)

• Figure 1: Overview of the LLM belief vulnerability study. Left (SMCR model): Persuasion strategies derived from the Source--Message--Channel--Receiver framework. Source strategies leverage authority or in-group attribution; message strategies use polite or statistical framing; receiver strategies manipulate self-esteem or confirmation bias via system prompts (channel manipulations are not applied). Center (persuasion pipeline): Multi-turn evaluation flow including an initial belief check (Turn 0), successive persuasive interactions with misinformation, implicit belief checks tracking confidence decay, and a final belief assessment (Turn 5). Top right (meta-cognition test): Comparison between standard prompting (RQ1) and meta-cognition prompting that elicits self-reported confidence (RQ2). Bottom right (adversarial training): Fine-tuning setup for RQ3, training models to recognize persuasive tactics while preserving factually correct responses.
• Figure 2: Confidence trajectories grouped by ending turn. Robust responses (turn 6) maintain high confidence; earlier-flipping responses show progressive decay. Lower initial confidence predicts vulnerability.
• Figure 3: Aggregated confidence score trajectories across persuasion rounds. The visualization shows how confidence scores progressively decrease before belief change occurs, with distinct patterns emerging based on initial confidence levels.
• Figure 4: Example for the use of top logprobs to know the log probabilities of the potential options for targeted tokens.
• Figure 5: ACC and MR trajectories for GPT-4o-mini across three datasets and four appeal types.