How Do Solidity Versions Affect Vulnerability Detection Tools? An Empirical Study

TL;DR

Solidity version updates can alter how vulnerability detection tools interpret and analyze smart contracts. The authors propose an empirical, exploratory study leveraging the SmartBugs framework and a large real-world dataset to evaluate tool compatibility with Solidity pragma directives, detection effectiveness, and execution time across versions from $0.4.x$ to $0.8.x$. They build a wild dataset, integrate multiple tools, perform manual validation and vulnerability injection, and apply statistical tests to compare static and dynamic analyses. The study aims to reveal gaps in tool support, guide practitioners in tool selection, and motivate improvements in vulnerability detectors as the Solidity language evolves.

Abstract

Context: Smart contract vulnerabilities pose significant security risks for the Ethereum ecosystem, driving the development of automated tools for detection and mitigation. Smart contracts are written in Solidity, a programming language that is rapidly evolving to add features and improvements to enhance smart contract security. New versions of Solidity change the compilation process, potentially affecting how tools interpret and analyze smart contract code. Objective: In such a continuously evolving landscape, we aim to investigate the compatibility of detection tools with Solidity versions. More specifically, we present a plan to study detection tools by empirically assessing (i) their compatibility with the Solidity pragma directives, (ii) their detection effectiveness, and (iii) their execution time across different versions of Solidity. Method: We will conduct an exploratory study by running several tools and collecting a large number of real-world smart contracts to create a balanced dataset. We will track and analyze the tool execution through SmartBugs, a framework that facilitates the tool execution and allows the integration of new tools.

Figures (2)

• Figure 1: Underflow Check on Different Solidity Versions.
• Figure 2: Overview of the Research Method.