DPBench: Large Language Models Struggle with Simultaneous Coordination

TL;DR

DPBench tackles whether large language models can coordinate in multi-agent settings when accessing shared resources. It encodes this challenge using the Dining Philosophers problem across eight experimental conditions to stress-test simultaneous versus sequential decision-making with and without communication. Across GPT-5.2, Claude Opus 4.5, and Grok 4.1, results reveal a strong asymmetry: models coordinate well in sequential settings but exhibit high deadlock under simultaneous decisions, with deadlock rates reaching up to $0.95$ (i.e., 95%) in some configurations and communication not reliably reducing deadlock. The findings suggest that external coordination mechanisms or explicit turn-taking protocols may be necessary for reliable concurrent resource access in multi-agent LLM systems, and DPBench is released as an open-source benchmark to accelerate progress.

Abstract

Large language models are increasingly deployed in multi-agent systems, yet we lack benchmarks that test whether they can coordinate under resource contention. We introduce DPBench, a benchmark based on the Dining Philosophers problem that evaluates LLM coordination across eight conditions that vary decision timing, group size, and communication. Our experiments with GPT-5.2, Claude Opus 4.5, and Grok 4.1 reveal a striking asymmetry: LLMs coordinate effectively in sequential settings but fail when decisions must be made simultaneously, with deadlock rates exceeding 95\% under some conditions. We trace this failure to convergent reasoning, where agents independently arrive at identical strategies that, when executed simultaneously, guarantee deadlock. Contrary to expectations, enabling communication does not resolve this problem and can even increase deadlock rates. Our findings suggest that multi-agent LLM systems requiring concurrent resource access may need external coordination mechanisms rather than relying on emergent coordination. DPBench is released as an open-source benchmark. Code and benchmark are available at https://github.com/najmulhasan-code/dpbench.

Figures (9)

• Figure 1: Deadlock state in the Dining Philosophers problem ($N=5$). Each philosopher holds one of their two adjacent forks (green) but needs the other to eat. That fork is held by their neighbor (red dashed), forming a circular wait: P$_0$$\to$P$_4$$\to$P$_3$$\to$P$_2$$\to$P$_1$$\to$P$_0$. No agent can proceed. This is the coordination failure DPBench measures.
• Figure 2: GPT-5.2 deadlock rates across all eight DPBench conditions. Simultaneous mode (orange) produces dramatically higher deadlock rates than sequential mode (blue). The gap is most pronounced with 3 philosophers, where simultaneous mode reaches 95--100% deadlock while sequential mode stays near 0%.
• Figure 3: Cross-model comparison of deadlock rates. GPT-5.2 (blue) achieves 0% deadlock in sequential mode, while Claude 4.5 (orange) and Grok 4.1 (green) still deadlock 60% and 25% of episodes respectively. All models struggle in simultaneous mode, with deadlock rates between 25--70%.
• Figure 4: Effect of communication on deadlock rates. Contrary to expectations, enabling communication (pink) often increases deadlock compared to no communication (blue). In simultaneous mode with 5 philosophers, deadlock rises from 25% to 65%. Sequential mode remains near 0% regardless of communication.
• Figure 5: Convergent reasoning leading to deadlock. All three philosophers independently decide to follow a "consistent order" by grabbing their right fork first. This identical reasoning produces identical actions, resulting in immediate deadlock.