Multi-Agent Teams Hold Experts Back

TL;DR

This paper interrogates whether self-organizing, heterogeneous LLM teams can achieve strong synergy, defined as matching or surpassing the top-performing member. Through two parallel evaluations—controlled human psychology intellective tasks and frontier ML benchmarks—the authors show that such teams consistently fail to leverage differential expertise, even when the expert is explicitly identified, incurring relative synergy gaps up to 37.6%. The analysis disentangles failures in identifying the expert from failures in leveraging it, revealing that the bottleneck lies in integrative compromise and consensus-seeking behaviors that dilute expertise, especially as team size grows. Interestingly, this consensus tendency also provides robustness to adversarial inputs, highlighting a trade-off between aligning the team for manipulation resistance and effectively harnessing expertise. The work carries significant implications for designing multi-agent systems, suggesting that explicit role assignment or new training objectives may be necessary to realize true synergistic collaboration; it also provides an open-source teamwork evaluation harness for broader benchmarking.

Abstract

Multi-agent LLM systems are increasingly deployed as autonomous collaborators, where agents interact freely rather than execute fixed, pre-specified workflows. In such settings, effective coordination cannot be fully designed in advance and must instead emerge through interaction. However, most prior work enforces coordination through fixed roles, workflows, or aggregation rules, leaving open the question of how well self-organizing teams perform when coordination is unconstrained. Drawing on organizational psychology, we study whether self-organizing LLM teams achieve strong synergy, where team performance matches or exceeds the best individual member. Across human-inspired and frontier ML benchmarks, we find that -- unlike human teams -- LLM teams consistently fail to match their expert agent's performance, even when explicitly told who the expert is, incurring performance losses of up to 37.6%. Decomposing this failure, we show that expert leveraging, rather than identification, is the primary bottleneck. Conversational analysis reveals a tendency toward integrative compromise -- averaging expert and non-expert views rather than appropriately weighting expertise -- which increases with team size and correlates negatively with performance. Interestingly, this consensus-seeking behavior improves robustness to adversarial agents, suggesting a trade-off between alignment and effective expertise utilization. Our findings reveal a significant gap in the ability of self-organizing multi-agent teams to harness the collective expertise of their members.

Figures (29)

• Figure 1: Multi-agent teams fail to leverage expertise.Panel 1: Strong synergy. The bars show the performance of the team and the expert. The expert outperforming the team demonstrates the absence of strong synergy. Panel 2: Teams fail to leverage expertise and prioritize consensus. An illustrative conversation where the non-experts prioritize compromising over the expert's opinion (denoted by the robot with a PhD cap). Panel 3: Larger teams perform worse. We observe that as team size increases, expertise dilution becomes more severe, with team performance further approaching the member average.
• Figure 2: Concentrated Expertise Performance: Lost at Sea. Teams fail to match expert performance even when explicitly told who the expert is using aggressively-tuned prompts. The minimal improvement from Expert Not Mentioned to Reveal Expert indicates that leveraging expertise is the primary bottleneck, not identification. This trend is robust across multiple team configurations: 4 Haiku-3.5, 2 Haiku-3.5 + 2 GPT-4o-mini, and 4 GPT-4o-mini. Performance measured by L1 distance from expert ranking (lower is better; 10 seeds per team configuration $\times$ information condition). Information conditions are defined in Section \ref{['sec:experimental-conditions']}. Results for all tasks and expertise distributions are provided in Appendix \ref{['app:performance-gradients']}.
• Figure 3: Expertise Dilution Effect. NASA Moon Survival (Reveal Expert condition) shows ranking error increasing with team size across all model compositions (100% Anthropic, 50/50 mixed, 100% OpenAI). The consistent upward trend demonstrates that expertise dilution is robust to team composition. Error bars are $\pm$ SEM. Complete plots for all tasks in Appendix \ref{['app:expertise-dilution-plots']}.
• Figure 4: Strategic Persistence Meets Integrative Compromise. Transcript excerpt from NASA Moon Survival (Reveal Expert condition). Agent 1 (the expert) exhibits strategic persistence, providing domain-specific reasoning and explicitly declining to change their ranking. Despite this, non-expert agents respond with integrative compromise, proposing middle-ground positions rather than deferring to the expert's judgment. Examples of epistemic deference and epistemic flexibility can be found in Appendix \ref{['app:epistemic-deference-methodology']}.
• Figure 5: NASA Moon Survival. The expertise leveraging gap persists across all team compositions. Teams consistently underperform the best individual regardless of whether expertise is concentrated in one agent or distributed across multiple agents. Lower ranking error is better. No Information: no agent receives expertise-inducing information. Expert Not Mentioned: expert(s) have information but team is not told who. Reveal Expert: team is explicitly told which agent(s) have expertise. Best Individual: expert agent queried alone.