LLM-SmartAudit: Advanced Smart Contract Vulnerability Detection

TL;DR

Experimental results indicate that the LLM-SmartAudit solution outperforms all traditional smart contract auditing tools, offering higher accuracy and greater efficiency, and can detect complex logic vulnerabilities that traditional tools have previously overlooked.

Abstract

The immutable nature of blockchain technology, while revolutionary, introduces significant security challenges, particularly in smart contracts. These security issues can lead to substantial financial losses. Current tools and approaches often focus on specific types of vulnerabilities. However, a comprehensive tool capable of detecting a wide range of vulnerabilities with high accuracy is lacking. This paper introduces LLM-SmartAudit, a novel framework leveraging the advanced capabilities of Large Language Models (LLMs) to detect and analyze vulnerabilities in smart contracts. Using a multi-agent conversational approach, LLM-SmartAudit employs a collaborative system with specialized agents to enhance the audit process. To evaluate the effectiveness of LLM-SmartAudit, we compiled two distinct datasets: a labeled dataset for benchmarking against traditional tools and a real-world dataset for assessing practical applications. Experimental results indicate that our solution outperforms all traditional smart contract auditing tools, offering higher accuracy and greater efficiency. Furthermore, our framework can detect complex logic vulnerabilities that traditional tools have previously overlooked. Our findings demonstrate that leveraging LLM agents provides a highly effective method for automated smart contract auditing.

Figures (5)

• Figure 1: Muliti-agent Conversation Framework
• Figure 2: Thought-Reasoning and Buffer-Reasoning Prompting Strategies
• Figure 3: Task Queue in BA mode and TA mode
• Figure 4: Collaborative Decision-Making between Two Agents
• Figure 5: Examples for Different Question-Answering