Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Belief in Authority: Impact of Authority in Multi-Agent Evaluation Framework

Junhyuk Choi, Jeongyoun Kwon, Heeju Kim, Haeun Cho, Hayeong Jung, Sehee Min, Bugeun Kim

TL;DR

This work investigates how role-based authority shapes interactions in multi-agent evaluation using ChatEval and French and Raven's power taxonomy. It conducts two experiments—free-form and content-controlled—to observe bias across 12-turn dialogues between a General Public agent and authority roles implemented as Judge, Foreman, Manager, Supervisor, Leader, Mentor, Specialist, Expert, and Attorney. The study finds that Expert and Referent roles exert stronger influence than Legitimate roles, and that authority bias arises from authoritative roles maintaining positions while general agents remain flexible; neutral responses suppress the bias. The results yield practical guidance for designing MAS with asymmetric interaction patterns and underscore the need for bias mitigation strategies in authority-enabled AI collaboration.

Abstract

Multi-agent systems utilizing large language models often assign authoritative roles to improve performance, yet the impact of authority bias on agent interactions remains underexplored. We present the first systematic analysis of role-based authority bias in free-form multi-agent evaluation using ChatEval. Applying French and Raven's power-based theory, we classify authoritative roles into legitimate, referent, and expert types and analyze their influence across 12-turn conversations. Experiments with GPT-4o and DeepSeek R1 reveal that Expert and Referent power roles exert stronger influence than Legitimate power roles. Crucially, authority bias emerges not through active conformity by general agents, but through authoritative roles consistently maintaining their positions while general agents demonstrate flexibility. Furthermore, authority influence requires clear position statements, as neutral responses fail to generate bias. These findings provide key insights for designing multi-agent frameworks with asymmetric interaction patterns.

Belief in Authority: Impact of Authority in Multi-Agent Evaluation Framework

TL;DR

This work investigates how role-based authority shapes interactions in multi-agent evaluation using ChatEval and French and Raven's power taxonomy. It conducts two experiments—free-form and content-controlled—to observe bias across 12-turn dialogues between a General Public agent and authority roles implemented as Judge, Foreman, Manager, Supervisor, Leader, Mentor, Specialist, Expert, and Attorney. The study finds that Expert and Referent roles exert stronger influence than Legitimate roles, and that authority bias arises from authoritative roles maintaining positions while general agents remain flexible; neutral responses suppress the bias. The results yield practical guidance for designing MAS with asymmetric interaction patterns and underscore the need for bias mitigation strategies in authority-enabled AI collaboration.

Abstract

Multi-agent systems utilizing large language models often assign authoritative roles to improve performance, yet the impact of authority bias on agent interactions remains underexplored. We present the first systematic analysis of role-based authority bias in free-form multi-agent evaluation using ChatEval. Applying French and Raven's power-based theory, we classify authoritative roles into legitimate, referent, and expert types and analyze their influence across 12-turn conversations. Experiments with GPT-4o and DeepSeek R1 reveal that Expert and Referent power roles exert stronger influence than Legitimate power roles. Crucially, authority bias emerges not through active conformity by general agents, but through authoritative roles consistently maintaining their positions while general agents demonstrate flexibility. Furthermore, authority influence requires clear position statements, as neutral responses fail to generate bias. These findings provide key insights for designing multi-agent frameworks with asymmetric interaction patterns.
Paper Structure (42 sections, 1 equation, 1 figure, 5 tables)

This paper contains 42 sections, 1 equation, 1 figure, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multi-Agent Role
  4. Authority Bias
  5. Experiment1 : Free Form Experiment
  6. Design of Authoritative Roles
  7. Generation of Chat Data
  8. Experimental Procedure
  9. Evaluation
  10. Selected Models
  11. Result and Discussion
  12. Model
  13. Neutral Option
  14. Role
  15. Turn
  16. ...and 27 more sections

Figures (1)

  • Figure 1: Overview of experimental framework with authoritative roles classified into three power types and 12-turn conversations between General Public and Authoritative role agents.