Detecting Buggy Contracts via Smart Testing

TL;DR

An interactive, self-deciding foundation model based system, called SmartSys, to support hybrid smart contract dynamic analyses, to teach foundation models about performance bottlenecks of different dynamic analysis techniques, making it possible to forecast the right technique and generates effective fuzz targets that can reach deep, hidden bugs.

Abstract

Smart contracts are susceptible to critical vulnerabilities. Hybrid dynamic analyses, such as concolic execution assisted fuzzing and foundation model assisted fuzzing, have emerged as highly effective testing techniques for smart contract bug detection recently. This hybrid approach has shown initial promise in real-world benchmarks, but it still suffers from low scalability to find deep bugs buried in complex code patterns. We observe that performance bottlenecks of existing dynamic analyses and model hallucination are two main factors limiting the scalability of this hybrid approach in finding deep bugs. To overcome the challenges, we design an interactive, self-deciding foundation model based system, called SmartSys, to support hybrid smart contract dynamic analyses. The key idea is to teach foundation models about performance bottlenecks of different dynamic analysis techniques, making it possible to forecast the right technique and generates effective fuzz targets that can reach deep, hidden bugs. To prune hallucinated, incorrect fuzz targets, SmartSys feeds foundation models with feedback from dynamic analysis during compilation and at runtime. The interesting results of SmartSys include: i) discovering a smart contract protocol vulnerability that has escaped eleven tools and survived multiple audits for over a year; ii) improving coverage by up to 14.3\% on real-world benchmarks compared to the baselines.

Figures (10)

• Figure 1: Listing \ref{['concolic_example']} fuzz targets. Hallucinated fuzz target at the top uses an imaginary loop and incorrect syntax; after a few iterations of fixing, partially corrected fuzz target still cannot reach the bug quickly. Eventually corrected fuzz target is syntactic and semantically correct with seeding values that can overcome plateaus quickly.
• Figure 2: Listing \ref{['dataflow']} fuzz targets. Hallucinated fuzz target initially includes imaginary libraries and incorrect function signature. After a few iterations of fixing, correct fuzz target at the bottom tells the fuzzer to start with bug reachable seeding values in given function order, and then to mutate input variables in the function deposit via test_deposit.
• Figure 3: SmartSys's workflow
• Figure 4: Code coverage comparisons of SmartSys with similar tools on the three benchmarks of D1.
• Figure 5: The effect of the hallucination suppression algorithm and interactive forecast and generator models on the three benchmarks of D1: The top row shows the effect of the hallucination suppression algorithm; the bottom row shows the effect of interactive models.