Phase Transition for Budgeted Multi-Agent Synergy

TL;DR

The paper develops a minimal, calibratable theory of budgeted multi-agent synergy under three binding constraints: finite context windows, lossy inter-agent communication, and shared failures. It introduces a compact set of environment parameters—single-agent scaling exponent $β$, communication fidelity $γ(m)$, and shared-error correlation $ρ$—to predict when scaling out yields signal amplification, saturation, or collapse, culminating in a sharp phase transition governed by $α_ρ$. In the amplifying regime, it defines an organization exponent $s$ and proves that budgeted synergy occurs when $s>β$, with closed-form compute allocations and explicit budget thresholds, while also characterizing finite-depth saturation via a fixed point and mixing depth. The framework is validated through controlled synthetic simulations and aligned with findings from large-scale matched-budget studies, offering design diagnostics and practical guidance for topology choice, budget allocation, and protocol improvements. Overall, the work shifts agent-system design from heuristic trial-and-error toward principled regime analysis, highlighting that hierarchy helps only when fidelity is sufficient and shared failures are weak under fixed budgets.

Abstract

Multi-agent systems can improve reliability, yet under a fixed inference budget they often help, saturate, or even collapse. We develop a minimal and calibratable theory that predicts these regimes from three binding constraints of modern agent stacks: finite context windows, lossy inter-agent communication, and shared failures among similar agents. Each leaf agent is summarized by a compute-performance scaling exponent $β$; communication is captured by a message-length fidelity curve $γ(m)$; dependence is captured by an effective shared-error correlation $ρ$; and a context window $W$ imposes hard fan-in limits that make hierarchy necessary. For binary success/failure tasks with majority aggregation, we prove a sharp phase transition for deep $b$-ary trees with correlated inputs and lossy communication: a single scalar $α_ρ$ (combining $γ(m)$, $ρ$, and fan-in $b$) determines whether weak signal is amplified to a nontrivial fixed point or washed out to chance. In the amplifying regime, we derive an organization exponent $s$ and show that budgeted synergy, i.e., outperforming the best single agent under the same total budget, occurs exactly when $s>β$, yielding closed-form compute allocation rules and explicit budget thresholds. We further characterize saturation via a mixing depth and provide a conservative clipped predictor that remains accurate across growth and saturation. A continuous-performance warm-up gives closed-form risks for star, chain, and tree organizations, making correlation- and communication-induced floors explicit and exposing the core design trade-offs in a smooth setting. Finally, we validate the predicted phase boundaries in controlled synthetic simulations and show how the same mechanisms explain the dominant bottlenecks reported in recent large-scale matched-budget studies of LLM agent-system scaling.

Figures (12)

• Figure 1: Canonical organizations under a context window constraint $W$. A star requires the root to read all messages, so feasibility enforces $Nm \le W$. A hierarchical tree enforces the constraint locally ($bm \le W$), enabling many more leaves at the cost of multi-hop communication and repeated aggregation. Chains avoid fan-in bottlenecks but compound communication loss.
• Figure 2: A possible use of the framework. Estimate $\beta$ (single-agent scaling), $\rho$ (shared failures), and $\gamma(m)$ or $\sigma_c^2(m)$ (communication fidelity) in a given setting, then use the theory to map budgets $B$ and context windows $W$ to plausible organizations and message lengths.
• Figure 3: Conceptual behavior of the tree recursion \ref{['eq:tree_rec']}. Increasing depth reduces error rapidly at first (by repeatedly averaging), but improvement saturates at a floor $v^\star$ determined by communication distortion and shared correlation. Longer messages (larger $m$) reduce $\sigma_c^2(m)$, lowering the floor, but also reduce feasible fan-in due to the context constraint $bm \le W$.
• Figure 4: The majority aggregation map $f_b(u)$ for odd fan-in $b \in \{3, 5, 7, 11\}$. Each curve shows how input bias $u$ transforms into output bias after majority voting. All curves pass through $(0,0)$ and $(1,1)$, with slope $f_b'(0) > 1$ at the origin---majority amplifies weak signals. Larger fan-in produces stronger amplification, but the gain grows only as $O(\sqrt{b})$.
• Figure 5: The $\rho$-shared correlation model (Definition \ref{['def:rho_shared']}). With probability $\rho$, all $b$ child agents share a common vote $V$ (left), producing perfect within-group correlation. This captures "groupthink" where agents fail together. With probability $1-\rho$, agents vote independently given the truth $Y$ (right), allowing errors to cancel through aggregation. The parameter $\rho$ equals the pairwise correlation of signed correctness and can be estimated from logs.

Theorems & Definitions (20)

• Lemma 1: Basic properties of the majority map
• Definition 2: $\rho$-shared correlation model
• Lemma 3: Correlated majority map under the $\rho$-shared model
• Theorem 4: Phase transition for deep majority trees
• Theorem 5: Small-signal amplification band
• Theorem 6: Finite-depth convergence to the fixed point
• Proposition 7: Centralization dominates under unlimited context
• Proposition 8: Bias decays along a chain under lossy communication
• Theorem 9: Closed-form compute allocation in the growth regime
• Corollary 10: A budget threshold for scale-out synergy (growth regime)