MulVul: Retrieval-augmented Multi-Agent Code Vulnerability Detection via Cross-Model Prompt Evolution

TL;DR

MulVul introduces a retrieval-augmented multi-agent framework for broad-coverage code vulnerability detection that explicitly handles heterogeneous vulnerability patterns. It employs a coarse-to-fine Router-Detector architecture with groundings from a SCALE-based knowledge base and cross-model prompt evolution to automatically generate specialized prompts for category routing and fine-grained typing. Grounding via evidence retrieval and isolated detector reasoning reduces hallucinations and error propagation, yielding state-of-the-art performance on PrimeVul across 130 CWE types with a Macro-F1 of $34.79\%$, and a strong improvement over manual prompts by $51.6\%$. The approach demonstrates robust few-shot performance and scalability advantages, though it notes limitations in language scope and API cost.

Abstract

Large Language Models (LLMs) struggle to automate real-world vulnerability detection due to two key limitations: the heterogeneity of vulnerability patterns undermines the effectiveness of a single unified model, and manual prompt engineering for massive weakness categories is unscalable. To address these challenges, we propose \textbf{MulVul}, a retrieval-augmented multi-agent framework designed for precise and broad-coverage vulnerability detection. MulVul adopts a coarse-to-fine strategy: a \emph{Router} agent first predicts the top-$k$ coarse categories and then forwards the input to specialized \emph{Detector} agents, which identify the exact vulnerability types. Both agents are equipped with retrieval tools to actively source evidence from vulnerability knowledge bases to mitigate hallucinations. Crucially, to automate the generation of specialized prompts, we design \emph{Cross-Model Prompt Evolution}, a prompt optimization mechanism where a generator LLM iteratively refines candidate prompts while a distinct executor LLM validates their effectiveness. This decoupling mitigates the self-correction bias inherent in single-model optimization. Evaluated on 130 CWE types, MulVul achieves 34.79\% Macro-F1, outperforming the best baseline by 41.5\%. Ablation studies validate cross-model prompt evolution, which boosts performance by 51.6\% over manual prompts by effectively handling diverse vulnerability patterns.

Figures (6)

• Figure 1: Comparison between MulVul and existing LLM-based vulnerability detection methods. (a) Existing methods rely on fixed prompts and lack external grounding. (b) MulVul adopts a coarse-to-fine, retrieval-augmented multi-agent framework for multi-type vulnerability detection.
• Figure 2: Overview of MulVul for vulnerability detection. The router agent first selects top-$k$ candidate vulnerability categories, and category-specific detector agents then perform fine-grained identification with retrieved CWE-specific evidence.
• Figure 3: Illustration of the Cross-Model Prompt Evolution Process. The generator LLM $\mathcal{M}_{\text{evo}}$ (Claude ) proposes and mutates prompts, while the executor LLM $\mathcal{M}_{\text{exec}}$ (GPT-4o) evaluates their fitness.
• Figure 4: F1 score vs. CWE sample count. MulVul outperforms baselines across all data regimes, with the largest gains on few-shot CWEs.
• Figure 5: Per-CWE F1 vs. sample count (log scale). MulVul (red) consistently outperforms baselines, especially in the few-shot region (left) .