Execution-State-Aware LLM Reasoning for Automated Proof-of-Vulnerability Generation

TL;DR

This work tackles PoV generation for real-world vulnerabilities by integrating execution-grounded feedback with semantic reasoning in an LLM-driven agent, forming an iterative hypothesis–verification–refinement loop. The DrillAgent framework decomposes the task into vulnerability analysis, code instrumentation, path exploration, and crash triggering, with a Trace-to-Prompt translator converting runtime traces into source-level constraints. Empirical results on SEC-bench show DrillAgent achieving up to a $52.8\%$ improvement in validated PoV solutions over strong baselines under fixed budgets, while delivering a substantial set of variant PoVs and broad cross-project coverage. The approach demonstrates that grounding LLM reasoning in concrete execution is essential for reliably reproducing deep, complex vulnerabilities in real software systems.

Abstract

Proof-of-Vulnerability (PoV) generation is a critical task in software security, serving as a cornerstone for vulnerability validation, false positive reduction, and patch verification. While directed fuzzing effectively drives path exploration, satisfying complex semantic constraints remains a persistent bottleneck in automated exploit generation. Large Language Models (LLMs) offer a promising alternative with their semantic reasoning capabilities; however, existing LLM-based approaches lack sufficient grounding in concrete execution behavior, limiting their ability to generate precise PoVs. In this paper, we present DrillAgent, an agentic framework that reformulates PoV generation as an iterative hypothesis-verification-refinement process. To bridge the gap between static reasoning and dynamic execution, DrillAgent synergizes LLM-based semantic inference with feedback from concrete program states. The agent analyzes the target code to hypothesize inputs, observes execution behavior, and employs a novel mechanism to translate low-level execution traces into source-level constraints. This closed-loop design enables the agent to incrementally align its input generation with the precise requirements of the vulnerability. We evaluate DrillAgent on SEC-bench, a large-scale benchmark of real-world C/C++ vulnerabilities. Experimental results show that DrillAgent substantially outperforms state-of-the-art LLM agent baselines under fixed budget constraints, solving up to 52.8% more CVE tasks than the best-performing baseline. These results highlight the necessity of execution-state-aware reasoning for reliable PoV generation in complex software systems.

Figures (7)

• Figure 1: gpac.cve-2023-0760: vulnerable code snippets.
• Figure 2: The Overall Workflow of DrillAgent.
• Figure 3: Simplified crash trace generated by the VAAgent.
• Figure 4: Automated code instrumentation pipeline.
• Figure 5: Vulnerability-Type Distribution of Solved Tasks.