Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Line-level Semantic Structure Learning for Code Vulnerability Detection

Ziliang Wang, Ge Li, Jia Li, Yihong Dong, Yingfei Xiong, Zhi Jin

TL;DR

The CSLS model outperforms the state-of-the-art baselines in code vulnerability detection, achieving 70.57% accuracy on the Devign dataset and a 49.59% F1 score on the Reveal dataset, demonstrating the importance of preserving and utilizing code structure information to improve the performance of code vulnerability detection models.

Abstract

Unlike the flow structure of natural languages, programming languages have an inherent rigidity in structure and grammar.However, existing detection methods based on pre-trained models typically treat code as a natural language sequence, ignoring its unique structural information. This hinders the models from understanding the code's semantic and structual information.To address this problem, we introduce the Code Structure-Aware Network through Line-level Semantic Learning (CSLS), which comprises four components: code preprocessing, global semantic awareness, line semantic awareness, and line semantic structure awareness.The preprocessing step transforms the code into two types of text: global code text and line-level code text.Unlike typical preprocessing methods, CSLS retains structural elements such as newlines and indent characters to enhance the model's perception of code lines during global semantic awareness.For line semantics structure awareness, the CSLS network emphasizes capturing structural relationships between line semantics.Different from the structural modeling methods based on code blocks (control flow graphs) or tokens, CSLS uses line semantics as the minimum structural unit to learn nonlinear structural relationships, thereby improving the accuracy of code vulnerability detection.We conducted extensive experiments on vulnerability detection datasets from real projects. The CSLS model outperforms the state-of-the-art baselines in code vulnerability detection, achieving 70.57% accuracy on the Devign dataset and a 49.59% F1 score on the Reveal dataset.

Line-level Semantic Structure Learning for Code Vulnerability Detection

TL;DR

The CSLS model outperforms the state-of-the-art baselines in code vulnerability detection, achieving 70.57% accuracy on the Devign dataset and a 49.59% F1 score on the Reveal dataset, demonstrating the importance of preserving and utilizing code structure information to improve the performance of code vulnerability detection models.

Abstract

Unlike the flow structure of natural languages, programming languages have an inherent rigidity in structure and grammar.However, existing detection methods based on pre-trained models typically treat code as a natural language sequence, ignoring its unique structural information. This hinders the models from understanding the code's semantic and structual information.To address this problem, we introduce the Code Structure-Aware Network through Line-level Semantic Learning (CSLS), which comprises four components: code preprocessing, global semantic awareness, line semantic awareness, and line semantic structure awareness.The preprocessing step transforms the code into two types of text: global code text and line-level code text.Unlike typical preprocessing methods, CSLS retains structural elements such as newlines and indent characters to enhance the model's perception of code lines during global semantic awareness.For line semantics structure awareness, the CSLS network emphasizes capturing structural relationships between line semantics.Different from the structural modeling methods based on code blocks (control flow graphs) or tokens, CSLS uses line semantics as the minimum structural unit to learn nonlinear structural relationships, thereby improving the accuracy of code vulnerability detection.We conducted extensive experiments on vulnerability detection datasets from real projects. The CSLS model outperforms the state-of-the-art baselines in code vulnerability detection, achieving 70.57% accuracy on the Devign dataset and a 49.59% F1 score on the Reveal dataset.
Paper Structure (28 sections, 12 equations, 5 figures, 8 tables)

This paper contains 28 sections, 12 equations, 5 figures, 8 tables.

Table of Contents

  1. Introduction
  2. RATIONALE
  3. Control Flow Graph (CFG) based on Code Block
  4. Undirected Graphs based on Code Tokens
  5. The Novelty of Our Work based on Line Semantics
  6. Approach
  7. Data Preprocessing
  8. Line-level semantic and structure aware
  9. Global Semantic Awareness
  10. Loss Function
  11. Study Design
  12. Datesets
  13. Performance Metrics
  14. Baseline Methods
  15. Implementation
  16. ...and 13 more sections

Figures (5)

  • Figure 1: Existing preprocessing methods make the model focus only on the positional relationship between tokens, making it difficult to identify the structural relationship between code lines. CSLS facilitates the model in identifying code lines and learning relationships between line semantics.
  • Figure 2: The CSLS framework implements vulnerability detection by capturing three different semantics: (1) Line-level Structural Semantics (Step 3), (2) Sensitive-Line Semantics (Step 4), and (3) Global Semantics (Step 5).
  • Figure 3: A Transformer model is used to capture the semantic structural relationships between lines.
  • Figure 4: Comparison of vulnerability detection performance of different models on two datasets.
  • Figure 5: Comparison of vulnerability detection performance of different prep-rocessing process.