Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets

Dat Tran, Douwe Kiela

Abstract

Recent work reports strong performance from multi-agent LLM systems (MAS), but these gains are often confounded by increased test-time computation. When computation is normalized, single-agent systems (SAS) can match or outperform MAS, yet the theoretical basis and evaluation methodology behind this comparison remain unclear. We present an information-theoretic argument, grounded in the Data Processing Inequality, suggesting that under a fixed reasoning-token budget and with perfect context utilization, single-agent systems are more information-efficient. This perspective further predicts that multi-agent systems become competitive when a single agent's effective context utilization is degraded, or when more compute is expended. We test these predictions in a controlled empirical study across three model families (Qwen3, DeepSeek-R1-Distill-Llama, and Gemini 2.5), comparing SAS with multiple MAS architectures under matched budgets. We find that SAS consistently match or outperform MAS on multi-hop reasoning tasks when reasoning tokens are held constant. Beyond aggregate performance, we conduct a detailed diagnostic analysis of system behavior and evaluation methodology. We identify significant artifacts in API-based budget control (particularly in Gemini 2.5) and in standard benchmarks, both of which can inflate apparent gains from MAS. Overall, our results suggest that, for multi-hop reasoning tasks, many reported advantages of multi-agent systems are better explained by unaccounted computation and context effects rather than inherent architectural benefits, and highlight the importance of understanding and explicitly controlling the trade-offs between compute, context, and coordination in agentic systems.

Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets

Abstract

Recent work reports strong performance from multi-agent LLM systems (MAS), but these gains are often confounded by increased test-time computation. When computation is normalized, single-agent systems (SAS) can match or outperform MAS, yet the theoretical basis and evaluation methodology behind this comparison remain unclear. We present an information-theoretic argument, grounded in the Data Processing Inequality, suggesting that under a fixed reasoning-token budget and with perfect context utilization, single-agent systems are more information-efficient. This perspective further predicts that multi-agent systems become competitive when a single agent's effective context utilization is degraded, or when more compute is expended. We test these predictions in a controlled empirical study across three model families (Qwen3, DeepSeek-R1-Distill-Llama, and Gemini 2.5), comparing SAS with multiple MAS architectures under matched budgets. We find that SAS consistently match or outperform MAS on multi-hop reasoning tasks when reasoning tokens are held constant. Beyond aggregate performance, we conduct a detailed diagnostic analysis of system behavior and evaluation methodology. We identify significant artifacts in API-based budget control (particularly in Gemini 2.5) and in standard benchmarks, both of which can inflate apparent gains from MAS. Overall, our results suggest that, for multi-hop reasoning tasks, many reported advantages of multi-agent systems are better explained by unaccounted computation and context effects rather than inherent architectural benefits, and highlight the importance of understanding and explicitly controlling the trade-offs between compute, context, and coordination in agentic systems.

Paper Structure

This paper contains 81 sections, 11 equations, 3 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Budget-Controlled Evaluation:
  4. When and why MAS helps:
  5. SAS vs. MAS:
  6. Multi-agent collaboration mechanisms:
  7. Context length, degradation, and long-context utilization:
  8. Theoretical justification
  9. Context degradation
  10. Approach
  11. Tasks and Datasets
  12. Single-agent architecture
  13. SAS with Longer Thinking Variant (SAS-L).
  14. Multi-agent Architectures
  15. Evaluation Metric
  16. ...and 66 more sections

Figures (3)

  • Figure 1: Sequential is the closest MAS comparator to SAS. Both are serial reasoning pipelines over the same question and the same global budget. The essential difference is that Sequential MAS externalizes intermediate reasoning into explicit messages between steps, whereas SAS keeps those intermediates latent within one continuous reasoning trajectory.
  • Figure 2: MuSiQue 4-hop accuracy across Gemini model versions with unlimited thinking tokens.
  • Figure 3: Context degradation results on MuSiQue 4-hop with Qwen3-30B-A3B under a fixed 1000-token thinking budget. The x-axis is the degradation level ($\alpha$ for deletion, masking, and substitution; $k$ for distractors), and the y-axis is answer accuracy.