Will LLM-powered Agents Bias Against Humans? Exploring the Belief-Dependent Vulnerability

TL;DR

The paper investigates whether LLM-powered agents develop intergroup bias under minimal 'us-vs-them' cues and shows that a belief-dependent human-norm script can attenuate bias when interactants are framed as humans. It introduces Belief Poisoning Attack (BPA), with two forms—profile poisoning (BPA-PP) and memory poisoning (BPA-MP)—to suppress the human-norm script and reactivate outgroup bias toward humans. Through a controlled multi-agent minimal-group allocation simulation, BPA proves effective across payoff structures, and the authors propose practical mitigations at profile and memory boundaries. The work highlights a new attack surface on belief states and offers defense prototypes, aiming to inform safer, more robust designs for human-facing agent systems. Its implications stress the need for belied-aware safeguards in deployed AI agents to prevent manipulation of long-term identity beliefs.

Abstract

LLM-empowered agents can exhibit not only demographic bias (e.g., gender, religion) but also intergroup bias triggered by minimal "us" versus "them" cues. When this intergroup boundary aligns with an agent-human divide, the risk shifts from disparities among human demographic groups to a more fundamental group-level asymmetry, i.e., humans as a whole may be treated as the outgroup by agents. To examine this possibility, we construct a controlled multi-agent social simulation based on allocation decisions under explicit payoff trade-offs and find that agents exhibit a consistent intergroup bias under minimal group cues. Although this bias is attenuated when some counterparts are framed as humans, we attribute the attenuation to an implicit human-norm script that favors humans yet activates only when the agent believes a real human is present. This belief dependence creates a new attack surface. We therefore introduce a Belief Poisoning Attack (BPA) that corrupts persistent identity beliefs to suppress the human-norm script and reactivate outgroup bias toward humans, instantiated as profile poisoning at initialization (BPA-PP) and memory poisoning via optimized belief-refinement suffixes injected into stored reflections (BPA-MP). Finally, we discuss practical mitigation strategies for hardening current agent frameworks against BPA, highlighting feasible interventions at profile and memory boundaries. Extensive experiments demonstrate both the existence of agent intergroup bias and the severity of BPA across settings. Our goal in identifying these vulnerabilities is to inform safer agent design, not to enable real-world exploitation.

Figures (6)

• Figure 1: Agent bias: from differential treatment across human groups to a group-level asymmetry where agents may treat humans as a whole as the outgroup.
• Figure 2: Overview of the multi-agent minimal-group allocation experiment.
• Figure 3: Results of the multi-agent minimal-group allocation experiment. Significance marks follow: $p>0.1$ (ns), $0.1 \ge p > 0.05$ ($^\ast$), $0.05 \ge p > 0.01$ ($^{\ast\ast}$), and $p \le 0.01$ ($^{\ast\ast\ast}$).
• Figure 4: The framework of BPA-MP.
• Figure 5: Temporal evolution of mean choice columns (with uncertainty bands) across Early/Middle/Late interaction periods under AVA, AVH w/o A, AVH w BPA-PP, AVH w BPA-MP, and AVH w BPA-PP+MP.