Leveraging Large Language Models and Machine Learning for Smart Contract Vulnerability Detection

TL;DR

The paper addresses the need for robust vulnerability detection in smart contracts by comparing traditional machine learning (e.g., LSTM) with fine-tuned large language models (DistilBERT, BERT) on a labeled dataset of 2,217 contracts spanning RE, IO, TD, and DD vulnerabilities. It finds that transformer-based models, particularly DistilBERT, deliver strong macro and weighted performance, while LSTM and BERT also achieve high accuracy but exhibit issues like overfitting in some cases. Fine-tuning domain-specific data improves detection and generalization, highlighting the value of LLMs in capturing complex code patterns that conventional ML may miss. The study supports deploying a hybrid ML/LLM framework to enhance the security of blockchain ecosystems and informs future work on dataset expansion and model exploration to bolster robustness and scalability.

Abstract

As blockchain technology and smart contracts become widely adopted, securing them throughout every stage of the transaction process is essential. The concern of improved security for smart contracts is to find and detect vulnerabilities using classical Machine Learning (ML) models and fine-tuned Large Language Models (LLM). The robustness of such work rests on a labeled smart contract dataset that includes annotated vulnerabilities on which several LLMs alongside various traditional machine learning algorithms such as DistilBERT model is trained and tested. We train and test machine learning algorithms to classify smart contract codes according to vulnerability types in order to compare model performance. Having fine-tuned the LLMs specifically for smart contract code classification should help in getting better results when detecting several types of well-known vulnerabilities, such as Reentrancy, Integer Overflow, Timestamp Dependency and Dangerous Delegatecall. From our initial experimental results, it can be seen that our fine-tuned LLM surpasses the accuracy of any other model by achieving an accuracy of over 90%, and this advances the existing vulnerability detection benchmarks. Such performance provides a great deal of evidence for LLMs ability to describe the subtle patterns in the code that traditional ML models could miss. Thus, we compared each of the ML and LLM models to give a good overview of each models strengths, from which we can choose the most effective one for real-world applications in smart contract security. Our research combines machine learning and large language models to provide a rich and interpretable framework for detecting different smart contract vulnerabilities, which lays a foundation for a more secure blockchain ecosystem.

Figures (4)

• Figure 1: Customized layer structure of data flow diagram for LSTM, DistilBERT and BERT Models in our experiment.
• Figure 2: Training and validation accuracy plots for LSTM, DistilBERT, and BERT models over 20 epochs. All models demonstrate increasing accuracy during training, with BERT achieving the highest peak validation accuracy. However, BERT also exhibits the highest variance in validation accuracy.
• Figure 3: Training and validation loss plots for LSTM, DistilBERT, and BERT models over 20 epochs. While all models show a decreasing trend in validation loss, BERT consistently outperforms the other models in terms of minimum validation loss. However, BERT's validation loss also fluctuates more significantly, indicating potential instability in its learning process.
• Figure 4: Confusion matrices for LSTM, DistilBERT, and BERT models, visualizing the distribution of true and predicted labels. All models demonstrate a strong bias toward predicting the Timestamp Dependency (TD) class, with BERT showing the highest accuracy for the Reentrancy (RE) class.