MOS: Towards Effective Smart Contract Vulnerability Detection through Mixture-of-Experts Tuning of Large Language Models

TL;DR

This work presents MOS, a vulnerability-detection framework for smart contracts that leverages Mixture-of-Experts tuning (MOE-Tuning) on large language models. By conducting continual pre-training on contract data, constructing a high-quality MOE dataset, and employing a vulnerability-aware routing mechanism with specialized experts, MOS outputs both vulnerability detections and structured explanations. Empirical evaluations across four vulnerability types show MOS achieving higher accuracy and F1 scores than state-of-the-art baselines, while human and LLM evaluations confirm the quality of its explanations (correctness, completeness, conciseness) with solid inter-evaluator agreement. The approach enables flexible coverage of undefined vulnerability types, scalable specialization, and practical utility for auditors, with publicly released datasets and code to advance future research.

Abstract

Smart contract vulnerabilities pose significant security risks to blockchain systems, potentially leading to severe financial losses. Existing methods face several limitations: (1) Program analysis-based approaches rely on predefined patterns, lacking flexibility for new vulnerability types; (2) Deep learning-based methods lack explanations; (3) Large language model-based approaches suffer from high false positives. We propose MOS, a smart contract vulnerability detection framework based on mixture-of-experts tuning (MOE-Tuning) of large language models. First, we conduct continual pre-training on a large-scale smart contract dataset to provide domain-enhanced initialization. Second, we construct a high-quality MOE-Tuning dataset through a multi-stage pipeline combining LLM generation and expert verification for reliable explanations. Third, we design a vulnerability-aware routing mechanism that activates the most relevant expert networks by analyzing code features and their matching degree with experts. Finally, we extend the feed-forward layers into multiple parallel expert networks, each specializing in specific vulnerability patterns. We employ a dual-objective loss function: one for optimizing detection and explanation performance, and another for ensuring reasonable distribution of vulnerability types to experts through entropy calculation. Experiments show that MOS significantly outperforms existing methods with average improvements of 6.32% in F1 score and 4.80% in accuracy. The vulnerability explanations achieve positive ratings (scores of 3-4 on a 4-point scale) of 82.96%, 85.21% and 94.58% for correctness, completeness, and conciseness through human and LLM evaluation.

Figures (8)

• Figure 1: The Life Cycle of Smart Contract
• Figure 2: Motivation: A complex smart contract with four vulnerabilities
• Figure 3: The Overview of MOS.
• Figure 4: Prompt Design for Vulnerability Detection
• Figure 5: Performance of four optimal baselines in the different categories and MOS