Automated Vulnerability Detection in Source Code Using Deep Representation Learning

TL;DR

The paper tackles automated vulnerability detection in large-scale C/C++ code by learning deep representations directly from lexed source. It builds a vast, function-level dataset (> $12$ million functions) from SATE IV Juliet, Debian, and GitHub, labeling functions via multiple static analyzers and a strict deduplication pipeline, then trains CNN/RNN-based feature extractors on a fixed $156$-token lexicon with embedding dimension $k=13$ and $n=512$ CNN filters of size $m=9$. A random forest ensemble operates on the learned neural features, achieving superior performance over a bag-of-words baseline and surpassing static analyzers on Juliet data, while remaining scalable to large repositories. This work demonstrates the feasibility and practicality of deep representation learning for code vulnerability detection and points to future work in richer dynamic labeling and broader code-classification applications, with potential for rapid, large-scale code security scanning.$12$-million-function scale, $156$-token vocabulary, $k=13$ embedding, and $n=512$ filters are key quantitative anchors of the approach.

Abstract

Increasing numbers of software vulnerabilities are discovered every year whether they are reported publicly or discovered internally in proprietary code. These vulnerabilities can pose serious risk of exploit and result in system compromise, information leaks, or denial of service. We leveraged the wealth of C and C++ open-source code available to develop a large-scale function-level vulnerability detection system using machine learning. To supplement existing labeled vulnerability datasets, we compiled a vast dataset of millions of open-source functions and labeled it with carefully-selected findings from three different static analyzers that indicate potential exploits. The labeled dataset is available at: https://osf.io/d45bw/. Using these datasets, we developed a fast and scalable vulnerability detection tool based on deep feature representation learning that directly interprets lexed source code. We evaluated our tool on code from both real software packages and the NIST SATE IV benchmark dataset. Our results demonstrate that deep feature representation learning on source code is a promising approach for automated software vulnerability detection.

Figures (5)

• Figure 1: Illustration of our convolutional neural representation-learning approach to source code classification. Input source code is lexed into a token sequence of variable length $\ell$, embedded into a $\ell\times k$ representation, filtered by $n$ convolutions of size $m\times k$, and maxpooled along the sequence length to a feature vector of fixed size $n$. The embedding and convolutional filters are learned by weighted cross entropy loss from fully-connected classification layers. The learned $n$-dimensional feature vector is used as input to a random forest classifier, which improves performance compared to the neural network classifier alone.
• Figure 2: Precision versus recall of different ML approaches using our lexer representation on Debian and Github test data. Vulnerable functions make up 6.5% of the test data.
• Figure 3: SATE IV test data ROC, with true vulnerability labels, compared to the three static analyzers we considered. Vulnerable functions make up 43% of the test data.
• Figure 4: Performance of a multi-label CNN + RF classifier on Debian and Github data by vulnerability type (see Table \ref{['table:bug_types']}.)
• Figure 5: Screenshot from our interactive vulnerability detection demo. The convolutional feature activation map zhou2016 for a detected vulnerability is overlaid in red on the original code.