Accommodation and Epistemic Vigilance: A Pragmatic Account of Why LLMs Fail to Challenge Harmful Beliefs

TL;DR

The paper argues that LLMs' failures to challenge harmful beliefs arise from a pragmatic default to accommodate user content and a lack of epistemic vigilance. It shows that three factors—at-issueness, linguistic encoding, and source reliability—shape LLM accommodation in ways aligned with human data across Cancer-Myth, SAGE-Eval, and ELEPHANT benchmarks. It introduces two simple interventions, an explicit correction instruction and the 'wait a minute' discourse marker, which substantially improve performance while controlling false positives. The findings have practical implications for benchmark design and safety improvements, demonstrating that pragmatic framing can yield safer, more reliable LLM behavior without extensive retraining.

Abstract

Large language models (LLMs) frequently fail to challenge users' harmful beliefs in domains ranging from medical advice to social reasoning. We argue that these failures can be understood and addressed pragmatically as consequences of LLMs defaulting to accommodating users' assumptions and exhibiting insufficient epistemic vigilance. We show that social and linguistic factors known to influence accommodation in humans (at-issueness, linguistic encoding, and source reliability) similarly affect accommodation in LLMs, explaining performance differences across three safety benchmarks that test models' ability to challenge harmful beliefs, spanning misinformation (Cancer-Myth, SAGE-Eval) and sycophancy (ELEPHANT). We further show that simple pragmatic interventions, such as adding the phrase "wait a minute", significantly improve performance on these benchmarks while preserving low false-positive rates. Our results highlight the importance of considering pragmatics for evaluating LLM behavior and improving LLM safety.

Figures (12)

• Figure 1: Mean ($\pm 95\%$ CI) benchmark scores by each factor (H1a-H1c). Higher is better for SAGE-Eval and Cancer-Myth, and lower is better for ELEPHANT (r/AITA and Subjective Statements). Each factor results in significantly higher overall performance in the expected direction for all factors. Except for framing sycophancy on r/AITA, the social sycophancy findings hold for almost all models individually as well (details in Figure \ref{['fig:all_elephant_source']}). Full details are in Fig. \ref{['fig:all_6conds']}.
• Figure 2: Mean score ($\pm$ 95% CI) of interventions on Cancer-Myth, SAGE-Eval and ELEPHANT. Higher is better for Cancer-Myth and SAGE-Eval, closer to 0 is better for ELEPHANT. We find that both interventions overall yield significant improvements for Cancer-Myth, validation and indirectness. For framing sycophancy, Explicit leads to excessive challenging (negative scores). For SAGE-Eval, only Explicit yields significant improvements overall after controlling for PFR. Raw (uncontrolled) scores and OEQ results are in Figure \ref{['fig:moreresults']}.
• Figure 3: Mean safety score ($\pm$ 95% CI) by at-issueness. The baseline and interventions are comparable on at-issue prompts, but the interventions improve performance significantly on the not-at-issue prompts. Breakdown by model is in Fig. \ref{['fig:atissueeachmodelsag']}.
• Figure A1: Mean ($\pm95\%$ CI) LLM error rates by accommodation factor in our Study 0 replicating giunta2025presuppositions's study of accommodation factors in humans. Though the differences are not statistically significant (likely because the sample size is $n = 16$), we find that the overall trends correspond to their findings.
• Figure A2: Top: Performance by model across all six conditions on Cancer-Myth and SAGE-Eval. Bottom: Cancer-Myth performance aggregated by linguistic factors only. Means are reported in Tables \ref{['tab:cancer_pass_hier']} and \ref{['tab:sage_hier']}.