CooperBench: Why Coding Agents Cannot be Your Teammates Yet

TL;DR

CooperBench reveals a fundamental coordination bottleneck in modern coding agents: two agents collaborating on overlapping tasks perform substantially worse than a single agent handling both, exposing a coordination gap. By grounding 652 tasks across 12 libraries and 4 languages in verifiable pipelines, the study shows that misaligned plans, unverifiable commitments, and jammed or erroneous communication drive failures, even as directed efforts reduce merge conflicts. Emergent coordination patterns such as role division, resource division, and negotiation hint at latent capabilities that could be reinforced with targeted training and lightweight coordination protocols. The work provides an open, scalable benchmark and framework for measuring social intelligence in AI coding agents, with implications for how we design agents that can safely and effectively work alongside humans and other agents. It advocates shifting focus from raw capability to developing robust coordination mechanisms and richer collaboration modalities.

Abstract

Resolving team conflicts requires not only task-specific competence, but also social intelligence to find common ground and build consensus. As AI agents increasingly collaborate on complex work, they must develop coordination capabilities to function as effective teammates. Yet we hypothesize that current agents lack these capabilities. To test this, we introduce CooperBench, a benchmark of over 600 collaborative coding tasks across 12 libraries in 4 programming languages. Each task assigns two agents different features that can be implemented independently but may conflict without proper coordination. Tasks are grounded in real open-source repositories with expert-written tests. Evaluating state-of-the-art coding agents, we observe the curse of coordination: agents achieve on average 30% lower success rates when working together compared to performing both tasks individually. This contrasts sharply with human teams, where adding teammates typically improves productivity. Our analysis reveals three key issues: (1) communication channels become jammed with vague, ill-timed, and inaccurate messages; (2) even with effective communication, agents deviate from their commitments; and (3) agents often hold incorrect expectations about others' plans and communication. Through large-scale simulation, we also observe rare but interesting emergent coordination behavior including role division, resource division, and negotiation. Our research presents a novel benchmark for collaborative coding and calls for a shift from pursuing individual agent capability to developing social intelligence.

Figures (10)

• Figure 1: The CooperBench benchmark draws tasks for two agents from a pool of features with potential conflicts. The agents execute the tasks in their individual environments, communicating in real time to coordinate. Success is measured by whether the resulting code changes by both agents are compatible and pass the requirements for both features.
• Figure 2: An example feature pool based on DSPy GitHub repository. This feature pool has 6 features which can be implemented compatibly based on the repository state, but without coordination agents could conflict with each other.
• Figure 3: The CooperBench construction pipeline. Each task is carefully engineered by domain experts to ensure conflicts are realistic, resolvable, and representative of production software development challenges.
• Figure 4: Left: Under Coop setting, agents with different foundation models perform significantly worse than how they perform under Solo setting, except for Qwen3-30B-A3B-Instruct-2507, which performs bad under both settings. This Solo-Coop gap is what we call the "coordination gap". Right: The relationship between tasks' technical difficulties and Solo-Coop gap. The shaded area has a large middle section which shows that the coordination gap is larger for middle-level tasks than for tasks which are extremely easy or difficult.
• Figure 5: (a) Effect of inter-agent communication on cooperation success or lack thereof. All agents fail to use communication for improving cooperation success. (b) Communication substantially reduces naive merge conflicts across all models. (c) Communication overhead as a percentage of all execution events, broken down by message type. Models that communicate more (e.g., Claude Sonnet 4.5, GPT-5) show larger reductions in conflict rate.