Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Agent Code Verification via Information Theory

Shreshth Rajan

TL;DR

The paper introduces CodeX-Verify, a four-agent static verification framework (Correctness, Security, Performance, Style) for LLM-generated code, and provides an information-theoretic justification for multi-agent aggregation. It proves that, under conditional independence, the combined detection information exceeds any single agent due to submodularity of mutual information, and demonstrates diminishing returns as more agents are added. Empirically, it evaluates 99 samples with 15 configurations, showing a 39.7 percentage point improvement over single agents (72.4% average accuracy with four agents) and achieving 76.1% true positive rate with 50% false positives, matching state-of-the-art TPR while running statically in under 200 ms per sample. Real-world validation on Claude-generated patches and ablation studies corroborate the theoretical claims, and results suggest practical deployment options (e.g., using Correctness+Performance for high accuracy) along with future directions toward hybrid static-dynamic verification and multi-language support.

Abstract

LLMs generate buggy code: 29.6% of SWE-bench solved patches fail, 62% of BaxBench solutions have vulnerabilities, and existing tools only catch 65% of bugs with 35% false positives. We built CodeX-Verify, a multi-agent system that uses four specialized agents to detect different types of bugs. We prove mathematically that combining agents with different detection patterns finds more bugs than any single agent when the agents look for different problems, using submodularity of mutual information under conditional independence. Measuring agent correlation of rho = 0.05 to 0.25 confirms they detect different bugs. Testing on 99 code samples with verified labels shows our system catches 76.1% of bugs, matching the best existing method (Meta Prompt Testing: 75%) while running faster and without test execution. We tested all 15 agent combinations and found that using multiple agents improves accuracy by 39.7 percentage points (from 32.8% to 72.4%) compared to single agents, with diminishing returns of +14.9pp, +13.5pp, and +11.2pp for agents 2, 3, and 4, validating our theoretical model. The best two-agent combination (Correctness + Performance) reaches 79.3% accuracy. Testing on 300 real patches from Claude Sonnet 4.5 runs in under 200ms per sample, making this practical for production use.

Multi-Agent Code Verification via Information Theory

TL;DR

The paper introduces CodeX-Verify, a four-agent static verification framework (Correctness, Security, Performance, Style) for LLM-generated code, and provides an information-theoretic justification for multi-agent aggregation. It proves that, under conditional independence, the combined detection information exceeds any single agent due to submodularity of mutual information, and demonstrates diminishing returns as more agents are added. Empirically, it evaluates 99 samples with 15 configurations, showing a 39.7 percentage point improvement over single agents (72.4% average accuracy with four agents) and achieving 76.1% true positive rate with 50% false positives, matching state-of-the-art TPR while running statically in under 200 ms per sample. Real-world validation on Claude-generated patches and ablation studies corroborate the theoretical claims, and results suggest practical deployment options (e.g., using Correctness+Performance for high accuracy) along with future directions toward hybrid static-dynamic verification and multi-language support.

Abstract

LLMs generate buggy code: 29.6% of SWE-bench solved patches fail, 62% of BaxBench solutions have vulnerabilities, and existing tools only catch 65% of bugs with 35% false positives. We built CodeX-Verify, a multi-agent system that uses four specialized agents to detect different types of bugs. We prove mathematically that combining agents with different detection patterns finds more bugs than any single agent when the agents look for different problems, using submodularity of mutual information under conditional independence. Measuring agent correlation of rho = 0.05 to 0.25 confirms they detect different bugs. Testing on 99 code samples with verified labels shows our system catches 76.1% of bugs, matching the best existing method (Meta Prompt Testing: 75%) while running faster and without test execution. We tested all 15 agent combinations and found that using multiple agents improves accuracy by 39.7 percentage points (from 32.8% to 72.4%) compared to single agents, with diminishing returns of +14.9pp, +13.5pp, and +11.2pp for agents 2, 3, and 4, validating our theoretical model. The best two-agent combination (Correctness + Performance) reaches 79.3% accuracy. Testing on 300 real patches from Claude Sonnet 4.5 runs in under 200ms per sample, making this practical for production use.

Paper Structure

This paper contains 45 sections, 4 theorems, 10 equations, 3 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLM Code Generation and Verification
  4. Multi-Agent Systems for Software Engineering
  5. Static Analysis and Vulnerability Detection
  6. Ensemble Learning and Information Theory
  7. Theoretical Framework
  8. Problem Formulation
  9. Why Multi-Agent Works
  10. Diminishing Returns
  11. Weighted Aggregation
  12. Sample Complexity and Generalization
  13. Agent Selection
  14. Decision Function
  15. Theory Summary
  16. ...and 30 more sections

Key Result

Theorem 1

Assume agents $A_1, \ldots, A_n$ are pairwise conditionally independent given bug status $B$: $I(A_i; A_j | B) = 0$ for $i \neq j$. Then $f(S) = I(A_S; B)$ is monotone submodular (Krause & Guestrin, 2011), implying: when multiple agents are informative about $B$.

Figures (3)

  • Figure 1: System architecture: parallel multi-agent analysis.
  • Figure 2: TPR-FPR comparison. Our system achieves competitive TPR (76%) while operating via static analysis.
  • Figure 3: Multi-agent scaling with diminishing marginal returns.

Theorems & Definitions (8)

  • Theorem 1: Multi-Agent Information Advantage
  • proof
  • Theorem 2: Diminishing Returns via Submodularity
  • proof
  • Theorem 3: Sample Complexity Bound
  • proof
  • Theorem 4: Generalization Error Bound
  • proof