Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMs

TL;DR

HiddenBench formalizes and quantifies collective reasoning under distributed information in multi-agent LLMs. By building a theory-grounded 65-task benchmark and validating it with human groups and 15 frontier LLMs, it reveals a consistent gap where multi-agent systems under distributed information underperform single agents with full information, due to latent information asymmetry and insufficient information-surfacing mechanisms. Ablation studies show that forced disclosure or explicit epistemic prompts can significantly improve performance, but scaling model size or individual reasoning alone does not solve coordination failures. The work provides a reproducible pipeline for generating and validating distributed-information tasks and argues for training signals and coordination protocols that incentivize information probing and unshared evidence uncovering. Overall, HiddenBench offers a principled framework to diagnose and drive progress toward reliable collective reasoning in distributed-information multi-agent AI systems.

Abstract

Multi-agent systems built on large language models (LLMs) are expected to enhance decision-making by pooling distributed information, yet systematically evaluating this capability has remained challenging. We introduce HiddenBench, a 65-task benchmark grounded in the Hidden Profile paradigm, which isolates collective reasoning under distributed information from individual reasoning ability. Evaluating 15 frontier LLMs, we find that multi-agent LLMs achieve only 30.1% accuracy under distributed information, compared to 80.7% accuracy for single agents given complete information. We trace this gap to a systematic failure mode: agents cannot recognize or act under latent information asymmetry-they fail to reason about what others might know but have not yet expressed, leading to premature convergence on shared evidence while critical distributed facts remain unexplored. These failures persist across prompting strategies, communication depths, and group sizes-and worsen as groups scale. While some models (e.g., Gemini-2.5-Flash/Pro) outperform others, neither model scale nor individual reasoning accuracy reliably predicts collective performance. Our results identify failures in collective information exploration in decision-making as a key limitation of multi-agent LLMs, and provide a theory-grounded, reproducible framework for diagnosing collective reasoning failures.

Figures (3)

• Figure 1: Overview of the Hidden Profile paradigm. Agents receive shared information (black) and unshared information (color) without recognizing the asymmetry. Only by sharing unshared information can they identify the optimal decision—here, walking up the hill rather than taking the other options (the tunnel and the bridge). See Table \ref{['table:task']} for the actual information distribution.
• Figure 2: Automatic pipeline for scalable Hidden Profile task generation. GPT-4.1 generates candidate tasks, which are then tested under both Full and Hidden Profile conditions across 10 sessions each. Tasks that satisfy validation thresholds ($\geq$ 80% pre-discussion accuracy in the Full Profile condition; $\leq$ 20% in the Hidden Profile condition) are retained in HiddenBench. From 200 candidates, the pipeline produced 57 validated tasks (28.5% validation rate).
• Figure 3: Collective reasoning performance across 15 LLMs on HiddenBench. Bars show average accuracy across 65 tasks under the average rule. The rightmost columns display the improvement from interaction ($Y^{\text{post}} - Y^{\text{pre}}$) and the gap between collective reasoning and individual reasoning with full information ($Y^{\text{post}} - Y^{\text{full}}$). Models meeting strong collective reasoning criteria ($Y^{\text{full}} > 0.8$ and $Y^{\text{post}} - Y^{\text{pre}} > 0.4 \times (Y^{\text{full}} - Y^{\text{pre}})$) are highlighted in bold. Error bars indicate mean $\pm$ s.e.m.