Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Software Vulnerability Detection Using Transformer-based Models

Sameer Shaik, Zhen Huang, Daniela Stan Raicu, Jacob Furst

Abstract

Detecting software vulnerabilities is critical to ensuring the security and reliability of modern computer systems. Deep neural networks have shown promising results on vulnerability detection, but they lack the capability to capture global contextual information on vulnerable code. To address this limitation, we explore the application of transformers for C/C++ vulnerability detection. We use program slices that encapsulate key syntactic and semantic features of program code, such as API function calls, array usage, pointer manipulations, and arithmetic expressions. By leveraging transformers' capability to capture both local and global contextual information on vulnerable code, our work can identify vulnerabilities accurately. Combined with data balancing and hyperparameter fine-tuning, our work offers a robust and efficient approach to identifying vulnerable code with moderate resource usage and training time.

Efficient Software Vulnerability Detection Using Transformer-based Models

Abstract

Detecting software vulnerabilities is critical to ensuring the security and reliability of modern computer systems. Deep neural networks have shown promising results on vulnerability detection, but they lack the capability to capture global contextual information on vulnerable code. To address this limitation, we explore the application of transformers for C/C++ vulnerability detection. We use program slices that encapsulate key syntactic and semantic features of program code, such as API function calls, array usage, pointer manipulations, and arithmetic expressions. By leveraging transformers' capability to capture both local and global contextual information on vulnerable code, our work can identify vulnerabilities accurately. Combined with data balancing and hyperparameter fine-tuning, our work offers a robust and efficient approach to identifying vulnerable code with moderate resource usage and training time.

Paper Structure

This paper contains 41 sections, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Vulnerability Detection using Neural Networks
  4. VulDeePecker
  5. LineVul
  6. SySeVR
  7. Program Slicing and Transformer-Based Models for Vulnerability Detection
  8. Self-Attention-Based Vulnerability Detection
  9. Transformer-Based Approaches for Vulnerability Detection
  10. VulBERTa
  11. Methodology
  12. Down Sampling Techniques
  13. Hypothesis 1: Equal Distribution by Subclass
  14. Hypothesis 2: Balance by Smallest Subclass
  15. Transformer Models for Vulnerability Detection
  16. ...and 26 more sections

Figures (5)

  • Figure 1: Down-sampling under Hypothesis 1: the numbers in orange refer to vulnerable samples, while the numbers in green refer to non-vulnerable samples.
  • Figure 2: Down-sampling under Hypothesis 2: the numbers in orange refer to vulnerable samples, while the numbers in green refer to non-vulnerable samples.
  • Figure 3: Trained on 28k and tested on remaining data
  • Figure 4: Trained on 28k and tested on remaining data
  • Figure 5: Trained on 44k and tested on 10k