Do Androids Dream of Unseen Puppeteers? Probing for a Conspiracy Mindset in Large Language Models

TL;DR

The paper investigates whether large language models exhibit conspiratorial tendencies and how sociodemographic conditioning and prompting influence such reasoning. It adapts validated psychometric conspiracy surveys into prompts and evaluates innate, demographic, and conditioned conspiratorial responses across open-weight LLMs. Key findings show partial alignment with conspiracy beliefs in baseline models, demographic conditioning can bias results, and targeted conspiracy prompts robustly shift models toward conspiratorial responses, with safety implications for deployment. The work advances computational social science by using AI as a proxy to study high-level cognitive constructs while highlighting risks of manipulation and the need for mitigation strategies.

Abstract

In this paper, we investigate whether Large Language Models (LLMs) exhibit conspiratorial tendencies, whether they display sociodemographic biases in this domain, and how easily they can be conditioned into adopting conspiratorial perspectives. Conspiracy beliefs play a central role in the spread of misinformation and in shaping distrust toward institutions, making them a critical testbed for evaluating the social fidelity of LLMs. LLMs are increasingly used as proxies for studying human behavior, yet little is known about whether they reproduce higher-order psychological constructs such as a conspiratorial mindset. To bridge this research gap, we administer validated psychometric surveys measuring conspiracy mindset to multiple models under different prompting and conditioning strategies. Our findings reveal that LLMs show partial agreement with elements of conspiracy belief, and conditioning with socio-demographic attributes produces uneven effects, exposing latent demographic biases. Moreover, targeted prompts can easily shift model responses toward conspiratorial directions, underscoring both the susceptibility of LLMs to manipulation and the potential risks of their deployment in sensitive contexts. These results highlight the importance of critically evaluating the psychological dimensions embedded in LLMs, both to advance computational social science and to inform possible mitigation strategies against harmful uses.

Figures (9)

• Figure 1: Structure of the simple prompt used in the RQ1 experiments.
• Figure 2: Structure of the persona prompt used in the RQ2 experiments.
• Figure 3: Average score of different LLMs' responses to conspiratorial items, grouped by clusters. The dashed line indicates a neutral stance on a Likert scale. The first five clusters are conspiracy-related, the last two are for control purposes. Error bars are C.I. 95%.
• Figure 4: Normalization effect, in percentage, of socio-demographic attributes on conspiratorial beliefs w.r.t the baseline, aggregated by all models and divided by socio-demographic target group. *:$p<.05$, **:$p<.01$, ***:$p<.001$. Error bars are C.I. 95%.
• Figure 5: Effect of the conspiracy conditioning compared against the baseline of conspiratorial mindset aggregated across all models. Error bars are C.I. 95%.