Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?

TL;DR

The paper investigates whether GitHub Copilot introduces vulnerabilities as often as human developers by leveraging the Big-Vul dataset to mirror vulnerability scenarios. Using a two-stage categorization and expert recategorization, it finds Copilot replicates the same vulnerability about 33% of the time and the corresponding fix about 25% of the time across 153 scenarios, with results varying by CWE type and vulnerability age. The approach reveals Copilot is not uniformly worse than humans in introducing vulnerabilities, but highlights risks when using Copilot for automated bug fixing. The findings have practical implications for dataset curation, targeted testing of code-generation tools, and cautious deployment of Copilot in security-sensitive contexts.

Abstract

Several advances in deep learning have been successfully applied to the software development process. Of recent interest is the use of neural language models to build tools, such as Copilot, that assist in writing code. In this paper we perform a comparative empirical analysis of Copilot-generated code from a security perspective. The aim of this study is to determine if Copilot is as bad as human developers. We investigate whether Copilot is just as likely to introduce the same software vulnerabilities as human developers. Using a dataset of C/C++ vulnerabilities, we prompt Copilot to generate suggestions in scenarios that led to the introduction of vulnerabilities by human developers. The suggestions are inspected and categorized in a 2-stage process based on whether the original vulnerability or fix is reintroduced. We find that Copilot replicates the original vulnerable code about 33% of the time while replicating the fixed code at a 25% rate. However this behaviour is not consistent: Copilot is more likely to introduce some types of vulnerabilities than others and is also more likely to generate vulnerable code in response to prompts that correspond to older vulnerabilities. Overall, given that in a significant number of cases it did not replicate the vulnerabilities previously introduced by human developers, we conclude that Copilot, despite performing differently across various vulnerability types, is not as bad as human developers at introducing vulnerabilities in code.

Figures (7)

• Figure 1: Overview of Methodology
• Figure 2: Overview of Scenario Re-creation
• Figure 3: Overview of Scenario Recategorization
• Figure 4: Snapshot of website used by coders to vote on sample recategorization. Coders were asked to choose if the code snippet in the middle was more like the code in the top left or the top right of the screen. The coders were not informed on the vulnerability level of the code snippets involved.
• Figure 5: Code Snippets showing Copilot reproducing the buggy code (Category A) for CWE-20 (Improper input validation). Listing \ref{['cwe-20-buggy']} shows the original buggy file with the missing input validation. Listing \ref{['cwe-20-fixed']} shows the fixed version of the code with input validation inserted. The code in listing \ref{['cwe-20-prompt']} is used a prompt for Copilot which generates the output code in listing \ref{['cwe-20-response']}. Like the buggy code in listing \ref{['cwe-20-buggy']}, Copilot's output also does not contain the input validation required to avoid CWE-20.