Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RealVul: Can We Detect Vulnerabilities in Web Applications with LLM?

Di Cao, Yong Liao, Xiuwei Shang

TL;DR

RealVul is presented, the first LLM-based framework designed for PHP vulnerability detection, and demonstrates a significant improvement in both effectiveness and generalization compared to existing methods, effectively boosting the vulnerability detection capabilities of these models.

Abstract

The latest advancements in large language models (LLMs) have sparked interest in their potential for software vulnerability detection. However, there is currently a lack of research specifically focused on vulnerabilities in the PHP language, and challenges in extracting samples and processing persist, hindering the model's ability to effectively capture the characteristics of specific vulnerabilities. In this paper, we present RealVul, the first LLM-based framework designed for PHP vulnerability detection, addressing these issues. By vulnerability candidate detection methods and employing techniques such as normalization, we can isolate potential vulnerability triggers while streamlining the code and eliminating unnecessary semantic information, enabling the model to better understand and learn from the generated vulnerability samples. We also address the issue of insufficient PHP vulnerability samples by improving data synthesis methods. To evaluate RealVul's performance, we conduct an extensive analysis using five distinct code LLMs on vulnerability data from 180 PHP projects. The results demonstrate a significant improvement in both effectiveness and generalization compared to existing methods, effectively boosting the vulnerability detection capabilities of these models.

RealVul: Can We Detect Vulnerabilities in Web Applications with LLM?

TL;DR

RealVul is presented, the first LLM-based framework designed for PHP vulnerability detection, and demonstrates a significant improvement in both effectiveness and generalization compared to existing methods, effectively boosting the vulnerability detection capabilities of these models.

Abstract

The latest advancements in large language models (LLMs) have sparked interest in their potential for software vulnerability detection. However, there is currently a lack of research specifically focused on vulnerabilities in the PHP language, and challenges in extracting samples and processing persist, hindering the model's ability to effectively capture the characteristics of specific vulnerabilities. In this paper, we present RealVul, the first LLM-based framework designed for PHP vulnerability detection, addressing these issues. By vulnerability candidate detection methods and employing techniques such as normalization, we can isolate potential vulnerability triggers while streamlining the code and eliminating unnecessary semantic information, enabling the model to better understand and learn from the generated vulnerability samples. We also address the issue of insufficient PHP vulnerability samples by improving data synthesis methods. To evaluate RealVul's performance, we conduct an extensive analysis using five distinct code LLMs on vulnerability data from 180 PHP projects. The results demonstrate a significant improvement in both effectiveness and generalization compared to existing methods, effectively boosting the vulnerability detection capabilities of these models.

Paper Structure

This paper contains 27 sections, 5 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Existing Datasets
  4. Existing Approaches
  5. Method
  6. Vulnerability Candidate Detection
  7. Potential Vulnerability Localization
  8. Program Slicing
  9. Data Preprocessing
  10. Labeling
  11. Normalizing
  12. Deduplication
  13. Model Training
  14. Data Synthesis
  15. Evaluation
  16. ...and 12 more sections

Figures (5)

  • Figure 1: In the case of using vulnerability repair to build a dataset, the green part will be considered secure, and the red part will be considered vulnerable.
  • Figure 2: RealVul architecture overview.
  • Figure 3: The process of vulnerability candidate detection from a real-world PHP project. We identify potential vulnerability triggers and analyze the data flow and control flow through the source file's AST. The obtained code snippets are our samples.
  • Figure 4: Comparison of ablation study results with the visualization of results from the first two experiments.
  • Figure 5: Two sets of sample Cases obtained through vulnerability reapir and RealVul. We mark the data flow and potential vulnerability statements.