PoCo: Agentic Proof-of-Concept Exploit Generation for Smart Contracts

TL;DR

PoCo introduces an agentic AI framework that automatically converts auditor-written vulnerability descriptions into executable PoC exploits for smart contracts, integrating compilation and testing within Foundry. Through a ReAct-style loop and tool-augmented autonomy, PoCo outperforms single-pass prompting and workflow baselines across 23 real-world vulnerabilities, producing numerous well-formed and logically correct PoCs. The evaluation employs a novel Proof-of-Patch dataset linking vulnerability findings to patches and patches themselves, validating PoCs against ground-truth mitigations. The approach demonstrates substantial efficiency gains for audits and provides a reproducible, open dataset for the smart contract security community. Annotation detail levels are shown to influence PoC quality, informing best practices for vulnerability reporting.

Abstract

Smart contracts operate in a highly adversarial environment, where vulnerabilities can lead to substantial financial losses. Thus, smart contracts are subject to security audits. In auditing, proof-of-concept (PoC) exploits play a critical role by demonstrating to the stakeholders that the reported vulnerabilities are genuine, reproducible, and actionable. However, manually creating PoCs is time-consuming, error-prone, and often constrained by tight audit schedules. We introduce POCO, an agentic framework that automatically generates executable PoC exploits from natural-language vulnerability descriptions written by auditors. POCO autonomously generates PoC exploits in an agentic manner by interacting with a set of code-execution tools in a Reason-Act-Observe loop. It produces fully executable exploits compatible with the Foundry testing framework, ready for integration into audit reports and other security tools. We evaluate POCO on a dataset of 23 real-world vulnerability reports. POCO consistently outperforms the prompting and workflow baselines, generating well-formed and logically correct PoCs. Our results demonstrate that agentic frameworks can significantly reduce the effort required for high-quality PoCs in smart contract audits. Our contribution provides readily actionable knowledge for the smart contract security community.

Figures (7)

• Figure 1: Overview of PoCo's workflow. Starting with a smart contract project under audit, and a vulnerability annotation written by an auditor, PoCo autonomously crafts an executable PoC exploit. The auditor reviews and validates the PoC before submitting the vulnerability report, potentially receiving a monetary reward. The development team receives the PoC along with reproduction steps, enabling them to reproduce the issue and implement a patch efficiently.
• Figure 2: Vulnerability example, inspired from the audit competition code4rena 2023-04-caviar.
• Figure 3: PoCo task prompt for PoC generation.
• Figure 4: Key moments from the #091 oracle-manipulation trajectory: reconnaissance, incremental debugging, and the final passing PoC.
• Figure 5: Key moments from the #077 procedural run: the procedural description annotation keeps PoCo chasing the narrated NFT counts instead of producing a minimal PoC.