Co-RedTeam: Orchestrated Security Discovery and Exploitation with LLM Agents

TL;DR

Co-RedTeam tackles automated red teaming for software security by marrying security-grounded reasoning, code-aware analysis, execution-grounded feedback, and layered long-term memory in a two-stage workflow (discovery, then iterative exploitation). The orchestrator coordinates specialized agents across stages, enabling evidence-backed vulnerability hypotheses and reproducible exploitation in an isolated environment. Empirical results on CyBench, BountyBench, and CyberGym show substantial gains in exploitation success and vulnerability detection over baselines, with ablation studies confirming the critical roles of execution feedback, memory, and tool-enabled analysis. The approach demonstrates robust, generalizable cyber-assessment capabilities and outlines clear paths for continual improvement through memory and plan refinement.

Abstract

Large language models (LLMs) have shown promise in assisting cybersecurity tasks, yet existing approaches struggle with automatic vulnerability discovery and exploitation due to limited interaction, weak execution grounding, and a lack of experience reuse. We propose Co-RedTeam, a security-aware multi-agent framework designed to mirror real-world red-teaming workflows by integrating security-domain knowledge, code-aware analysis, execution-grounded iterative reasoning, and long-term memory. Co-RedTeam decomposes vulnerability analysis into coordinated discovery and exploitation stages, enabling agents to plan, execute, validate, and refine actions based on real execution feedback while learning from prior trajectories. Extensive evaluations on challenging security benchmarks demonstrate that Co-RedTeam consistently outperforms strong baselines across diverse backbone models, achieving over 60% success rate in vulnerability exploitation and over 10% absolute improvement in vulnerability detection. Ablation and iteration studies further confirm the critical role of execution feedback, structured interaction, and memory for building robust and generalizable cybersecurity agents.

Figures (4)

• Figure 1: Overview of Co-RedTeam. Co-RedTeam is a security-aware multi-agent framework for automatic vulnerability discovery and exploitation. (Top) Given a target codebase (and optional vulnerability hints), the orchestrator coordinates two stages. Stage I ( Vulnerability Discovery): Analysis and Critique agents discuss together leveraging code-browsing tools and security documentation to identify and validate candidate vulnerabilities with concrete evidence. Stage II (Iterative Exploitation): Planning, Validation, Execution, and Evaluation agents interact with an isolated execution environment to iteratively reproduce vulnerabilities through execution-grounded feedback. Throughout the process, a layered long-term memory stores vulnerability patterns, high-level strategies, and concrete technical actions, enabling experience reuse and continual improvement across tasks.
• Figure 2: Effect of maximum exploitation iterations. Success rate on CyBench versus maximum exploitation iteration, illustrating how Co-RedTeam benefits from iterative planning.
• Figure 3: Memory-driven performance evolution on CyberGym. Moving-average success rate (window size 100) tasks for four memory configurations using Gemini-2.5-Pro.
• Figure 4: Effect of maximum detection iterations. Success rate on CyBench versus maximum detection iteration, illustrating how Co-RedTeam benefits from multi-turn discussions.