Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Coopetition: Leveraging Coarse Verifier Signals for Resilient Multi-Agent LLM Reasoning

Rui Jerry Huang, Wendy Liu, Anastasia Miin, Lei Ding

TL;DR

This paper tackles inference-time reasoning for LLMs in multi-agent settings, where rigid coordination and self-bias can hinder performance. It introduces Adaptive Coopetition (AdCo), a lightweight multi-round framework that uses coarse verifier signals to adaptively switch between collaboration and competition among agents, guided by a revised UCB-like policy. Experiments on the DeepMath-103K dataset show a ~20% relative accuracy improvement over baselines with heterogeneous models, along with strong stability across data scales. Ablation studies reveal the necessity of the revised UCB policy and model diversity, while highlighting limitations such as computation costs and opportunities for improved state-aware exploration and aggregation in future work.

Abstract

Inference-time computation is a critical yet challenging paradigm for enhancing the reasoning performance of large language models (LLMs). While existing strategies improve reasoning stability and consistency, they suffer from notable limitations: self-correction often reinforces the model's initial biases, and Multi-Agent Collaboration (MAC) often fails due to the lack of efficient coordination mechanisms, leading to collective errors. Although high-performing verifiers can detect reasoning errors, making them reliable requires substantial training. To address these challenges, we introduce a novel inference-time framework, Adaptive Coopetition (AdCo), in which LLM agents utilize an adaptive, UCB-based "coopetition" mechanism. At each round, agents leverage coarse verifier signals to determine whether to collaborate or compete, and iteratively refine their reasoning based on peer feedback. Without relying on high-performance verifiers, our adaptive strategy achieves significant performance gains on mathematical reasoning benchmarks, yielding a 20% relative improvement over baselines on the more challenging dataset. Our approach remains robust and consistent in terms of accuracy under different sample sizes and configurations. This adaptive, signal-guided "coopetition" framework enhances reasoning robustness by leveraging both model knowledge diversity and reasoning trace measures, while also promoting uncertainty-driven exploration, especially when participants have comparable capabilities. From this perspective, our work offers a fresh lens on inference-time computation and paves the way for more resilient multi-agent LLM systems. Our code is available at: https://github.com/AdCo-Research/adaptive-coopetition.

Adaptive Coopetition: Leveraging Coarse Verifier Signals for Resilient Multi-Agent LLM Reasoning

TL;DR

This paper tackles inference-time reasoning for LLMs in multi-agent settings, where rigid coordination and self-bias can hinder performance. It introduces Adaptive Coopetition (AdCo), a lightweight multi-round framework that uses coarse verifier signals to adaptively switch between collaboration and competition among agents, guided by a revised UCB-like policy. Experiments on the DeepMath-103K dataset show a ~20% relative accuracy improvement over baselines with heterogeneous models, along with strong stability across data scales. Ablation studies reveal the necessity of the revised UCB policy and model diversity, while highlighting limitations such as computation costs and opportunities for improved state-aware exploration and aggregation in future work.

Abstract

Inference-time computation is a critical yet challenging paradigm for enhancing the reasoning performance of large language models (LLMs). While existing strategies improve reasoning stability and consistency, they suffer from notable limitations: self-correction often reinforces the model's initial biases, and Multi-Agent Collaboration (MAC) often fails due to the lack of efficient coordination mechanisms, leading to collective errors. Although high-performing verifiers can detect reasoning errors, making them reliable requires substantial training. To address these challenges, we introduce a novel inference-time framework, Adaptive Coopetition (AdCo), in which LLM agents utilize an adaptive, UCB-based "coopetition" mechanism. At each round, agents leverage coarse verifier signals to determine whether to collaborate or compete, and iteratively refine their reasoning based on peer feedback. Without relying on high-performance verifiers, our adaptive strategy achieves significant performance gains on mathematical reasoning benchmarks, yielding a 20% relative improvement over baselines on the more challenging dataset. Our approach remains robust and consistent in terms of accuracy under different sample sizes and configurations. This adaptive, signal-guided "coopetition" framework enhances reasoning robustness by leveraging both model knowledge diversity and reasoning trace measures, while also promoting uncertainty-driven exploration, especially when participants have comparable capabilities. From this perspective, our work offers a fresh lens on inference-time computation and paves the way for more resilient multi-agent LLM systems. Our code is available at: https://github.com/AdCo-Research/adaptive-coopetition.
Paper Structure (37 sections, 5 equations, 8 figures, 2 tables)

This paper contains 37 sections, 5 equations, 8 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Adaptive coopetition
  3. Coarse verifier signals:
  4. Model Diversity:
  5. Low-quality feedback isolation:
  6. Iterative adaptive coopetition:
  7. Experiments
  8. Performance evaluation
  9. Accuracy & stability:
  10. Model diversity:
  11. Efficiency:
  12. Ablation study
  13. Revised UCB-based action policy:
  14. Impact of agent capability:
  15. Limitations & future improvements
  16. ...and 22 more sections

Figures (8)

  • Figure 1: Overview of adaptive coopetition
  • Figure 2: AdCo shows clear improvement over baselines.
  • Figure 3: AdCo shows stability (STDEV < 1%) from 600-4000 data points.
  • Figure 4: Worker agent architecture: relying on Pub/sub channel to exchange information within the worker cluster, each agent individually carries out initial reasoning and continues iterative reasoning until the cluster consensus is reached via majority voting.
  • Figure 5: Initial reasoning prompt
  • ...and 3 more figures