Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Plan-Execute Framework for Smart Contract Security Auditing

Zhiyuan Wei, Jing Sun, Zijian Zhang, Zhe Hou, Zixiao Zhao

TL;DR

SmartAuditFlow presents a Plan-Execute framework for automated smart contract security auditing that dynamically generates adaptive audit plans and executes them through a structured, multi-stage workflow. By integrating iterative prompt optimization, static analysis grounding, and Retrieval-Augmented Generation, the system reduces LLM hallucinations and improves vulnerability detection across benchmarks. Empirical results show superiority over traditional tools and standalone LLMs, achieving 100% detection on common vulnerabilities, strong ranking performance (MRR/MAP), and complete CVE coverage in a 13-CVE benchmark, with notable gains when external knowledge sources are combined. The framework's modularity allows seamless swapping of LLM backbones and supports scalable, efficient auditing on real-world contracts, signaling a practical impact for automated blockchain security tools and future extensions in verification and knowledge-grounded reasoning.

Abstract

Large Language Models (LLMs) have shown great promise in code analysis and auditing; however, they still struggle with hallucinations and limited context-aware reasoning. We introduce SmartAuditFlow, a novel Plan-Execute framework that enhances smart contract security analysis through dynamic audit planning and structured execution. Unlike conventional LLM-based auditing approaches that follow fixed workflows and predefined steps, SmartAuditFlow dynamically generates and refines audit plans based on the unique characteristics of each smart contract. It continuously adjusts its auditing strategy in response to intermediate LLM outputs and newly detected vulnerabilities, ensuring a more adaptive and precise security assessment. The framework then executes these plans step by step, applying a structured reasoning process to enhance vulnerability detection accuracy while minimizing hallucinations and false positives. To further improve audit precision, SmartAuditFlow integrates iterative prompt optimization and external knowledge sources, such as static analysis tools and Retrieval-Augmented Generation (RAG). This ensures audit decisions are contextually informed and backed by real-world security knowledge, producing comprehensive security reports. Extensive evaluations across multiple benchmarks demonstrate that SmartAuditFlow outperforms existing methods, achieving 100 percent accuracy on common and critical vulnerabilities, 41.2 percent accuracy for comprehensive coverage of known smart contract weaknesses in real-world projects, and successfully identifying all 13 tested CVEs. These results highlight SmartAuditFlow's scalability, cost-effectiveness, and superior adaptability over traditional static analysis tools and contemporary LLM-based approaches, establishing it as a robust solution for automated smart contract auditing.

Adaptive Plan-Execute Framework for Smart Contract Security Auditing

TL;DR

SmartAuditFlow presents a Plan-Execute framework for automated smart contract security auditing that dynamically generates adaptive audit plans and executes them through a structured, multi-stage workflow. By integrating iterative prompt optimization, static analysis grounding, and Retrieval-Augmented Generation, the system reduces LLM hallucinations and improves vulnerability detection across benchmarks. Empirical results show superiority over traditional tools and standalone LLMs, achieving 100% detection on common vulnerabilities, strong ranking performance (MRR/MAP), and complete CVE coverage in a 13-CVE benchmark, with notable gains when external knowledge sources are combined. The framework's modularity allows seamless swapping of LLM backbones and supports scalable, efficient auditing on real-world contracts, signaling a practical impact for automated blockchain security tools and future extensions in verification and knowledge-grounded reasoning.

Abstract

Large Language Models (LLMs) have shown great promise in code analysis and auditing; however, they still struggle with hallucinations and limited context-aware reasoning. We introduce SmartAuditFlow, a novel Plan-Execute framework that enhances smart contract security analysis through dynamic audit planning and structured execution. Unlike conventional LLM-based auditing approaches that follow fixed workflows and predefined steps, SmartAuditFlow dynamically generates and refines audit plans based on the unique characteristics of each smart contract. It continuously adjusts its auditing strategy in response to intermediate LLM outputs and newly detected vulnerabilities, ensuring a more adaptive and precise security assessment. The framework then executes these plans step by step, applying a structured reasoning process to enhance vulnerability detection accuracy while minimizing hallucinations and false positives. To further improve audit precision, SmartAuditFlow integrates iterative prompt optimization and external knowledge sources, such as static analysis tools and Retrieval-Augmented Generation (RAG). This ensures audit decisions are contextually informed and backed by real-world security knowledge, producing comprehensive security reports. Extensive evaluations across multiple benchmarks demonstrate that SmartAuditFlow outperforms existing methods, achieving 100 percent accuracy on common and critical vulnerabilities, 41.2 percent accuracy for comprehensive coverage of known smart contract weaknesses in real-world projects, and successfully identifying all 13 tested CVEs. These results highlight SmartAuditFlow's scalability, cost-effectiveness, and superior adaptability over traditional static analysis tools and contemporary LLM-based approaches, establishing it as a robust solution for automated smart contract auditing.

Paper Structure

This paper contains 74 sections, 10 equations, 5 figures, 9 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Backgrounds
  3. Smart Contract Auditing
  4. Automated Vulnerability Detection
  5. Agentic Systems and Workflows
  6. Methodology
  7. Overview
  8. Plan Phase
  9. Initial Analysis and Adaptive Planning Stages (A1-A2)
  10. Iterative Prompt Optimization Framework
  11. Multi-Criteria Scoring Function $f(\rho, c, A)$
  12. Output Alignment Score $(f_{\text{exec}})$:
  13. Output Likelihood Score ($f_{\text{log}}$):
  14. Static Analysis Tool Involvement
  15. Execute Phase
  16. ...and 59 more sections

Figures (5)

  • Figure 1: Overview of the SmartAuditFlow system.
  • Figure 2: (a) Basic Calibration directly uses LLM to validate the initial review finding; (b) RAG-Enhanced Calibration integrates external knowledge for robust validation and comprises two primary components: Retriever and Generator.
  • Figure 3: Evaluation of Smart Contract Vulnerability Detection Tools - A Comparative Analysis
  • Figure 4: Trade-off Between Accuracy (top-max) and Efficiency (Average Number of Predictions)
  • Figure 5: Trade-off Between Accuracy (top-max) and Efficiency (Average Number of Predictions)