GPTScan: Detecting Logic Vulnerabilities in Smart Contracts by Combining GPT with Program Analysis

TL;DR

GPTScan presents a novel framework that fuses GPT-driven code understanding with static analysis to detect logic vulnerabilities in smart contracts. By decomposing vulnerabilities into code-level scenarios and properties and applying multi-dimensional filtering, it efficiently narrows candidate functions and uses static confirmation to validate GPT-derived findings. Evaluation on diverse real-world datasets shows strong precision for token contracts, solid recall for vulnerable contracts, and the discovery of new vulnerabilities, highlighting its value as a supplement to human auditors. The approach offers a practical, scalable solution with competitive speed and cost, and lays groundwork for broader coverage of logic vulnerabilities in DeFi smart contracts.

Abstract

Smart contracts are prone to various vulnerabilities, leading to substantial financial losses over time. Current analysis tools mainly target vulnerabilities with fixed control or data-flow patterns, such as re-entrancy and integer overflow. However, a recent study on Web3 security bugs revealed that about 80% of these bugs cannot be audited by existing tools due to the lack of domain-specific property description and checking. Given recent advances in Large Language Models (LLMs), it is worth exploring how Generative Pre-training Transformer (GPT) could aid in detecting logicc vulnerabilities. In this paper, we propose GPTScan, the first tool combining GPT with static analysis for smart contract logic vulnerability detection. Instead of relying solely on GPT to identify vulnerabilities, which can lead to high false positives and is limited by GPT's pre-trained knowledge, we utilize GPT as a versatile code understanding tool. By breaking down each logic vulnerability type into scenarios and properties, GPTScan matches candidate vulnerabilities with GPT. To enhance accuracy, GPTScan further instructs GPT to intelligently recognize key variables and statements, which are then validated by static confirmation. Evaluation on diverse datasets with around 400 contract projects and 3K Solidity files shows that GPTScan achieves high precision (over 90%) for token contracts and acceptable precision (57.14%) for large projects like Web3Bugs. It effectively detects ground-truth logic vulnerabilities with a recall of over 70%, including 9 new vulnerabilities missed by human auditors. GPTScan is fast and cost-effective, taking an average of 14.39 seconds and 0.01 USD to scan per thousand lines of Solidity code. Moreover, static confirmation helps GPTScan reduce two-thirds of false positives.

Figures (8)

• Figure 1: The Risky First Deposit (line 8-9) vulnerability.
• Figure 2: The Wrong Checkpoint Order (line 6-7 & line 10-11).
• Figure 3: A high-level overview of GPTScan, with blue blocks denoting GPT tasks and green blocks representing static analysis.
• Figure 4: Prompt for scenario and property matching.
• Figure 5: A prompt for finding related variables/statements.