Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Large Language Models and Simple, Stupid Bugs

Kevin Jesse, Toufique Ahmed, Premkumar T. Devanbu, Emily Morgan

TL;DR

The study investigates how large language models used for code completion, notably Codex, generate single-statement bugs (SStuBs) and how prompting strategies influence outcomes. Using the ManySStuBs4J Java dataset across Codex, PolyCoder, and CodeGen, it shows that LLMs produce about twice as many SStuBs as patches, but can avoid a meaningful fraction of bugs when prompted wisely. The most effective mitigation comes from automatically generated natural language comments inserted into prompts, which improve bug/patch ratios and patch rates, even when comments themselves contain minor inaccuracies. The findings highlight both the risk and potential of AI-assisted coding and point to practical practices, like automated commenting, to steer LLMs toward safer, more maintainable code.

Abstract

With the advent of powerful neural language models, AI-based systems to assist developers in coding tasks are becoming widely available; Copilot is one such system. Copilot uses Codex, a large language model (LLM), to complete code conditioned on a preceding "prompt". Codex, however, is trained on public GitHub repositories, viz., on code that may include bugs and vulnerabilities. Previous studies [1], [2] show Codex reproduces vulnerabilities seen in training. In this study, we examine how prone Codex is to generate an interesting bug category, single statement bugs, commonly referred to as simple, stupid bugs or SStuBs in the MSR community. We find that Codex and similar LLMs do help avoid some SStuBs, but do produce known, verbatim SStuBs as much as 2x as likely than known, verbatim correct code. We explore the consequences of the Codex generated SStuBs and propose avoidance strategies that suggest the possibility of reducing the production of known, verbatim SStubs, and increase the possibility of producing known, verbatim fixes.

Large Language Models and Simple, Stupid Bugs

TL;DR

The study investigates how large language models used for code completion, notably Codex, generate single-statement bugs (SStuBs) and how prompting strategies influence outcomes. Using the ManySStuBs4J Java dataset across Codex, PolyCoder, and CodeGen, it shows that LLMs produce about twice as many SStuBs as patches, but can avoid a meaningful fraction of bugs when prompted wisely. The most effective mitigation comes from automatically generated natural language comments inserted into prompts, which improve bug/patch ratios and patch rates, even when comments themselves contain minor inaccuracies. The findings highlight both the risk and potential of AI-assisted coding and point to practical practices, like automated commenting, to steer LLMs toward safer, more maintainable code.

Abstract

With the advent of powerful neural language models, AI-based systems to assist developers in coding tasks are becoming widely available; Copilot is one such system. Copilot uses Codex, a large language model (LLM), to complete code conditioned on a preceding "prompt". Codex, however, is trained on public GitHub repositories, viz., on code that may include bugs and vulnerabilities. Previous studies [1], [2] show Codex reproduces vulnerabilities seen in training. In this study, we examine how prone Codex is to generate an interesting bug category, single statement bugs, commonly referred to as simple, stupid bugs or SStuBs in the MSR community. We find that Codex and similar LLMs do help avoid some SStuBs, but do produce known, verbatim SStuBs as much as 2x as likely than known, verbatim correct code. We explore the consequences of the Codex generated SStuBs and propose avoidance strategies that suggest the possibility of reducing the production of known, verbatim SStubs, and increase the possibility of producing known, verbatim fixes.
Paper Structure (19 sections, 9 figures, 5 tables)

This paper contains 19 sections, 9 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Simple, Stupid Bugs
  4. Examining Codex Completions
  5. Methodology
  6. Experimental Setup
  7. Results
  8. SStuB production in LLMs (RQ1)
  9. Number of commits to fix LLM produced SStuBs (RQ2)
  10. SStuB regularity (RQ3)
  11. Avoiding SStuBs (RQ4)
  12. Commenting vs Traditional Prompting (RQ4 cont.)
  13. What if we insert a 'buggy' comment? (RQ5)
  14. Discussion
  15. Implications of Findings
  16. ...and 4 more sections

Figures (9)

  • Figure 1: The orange highlighted code is the candidate single line completion that Codex can match to the automatic evaluation either the known bug or fix. Blue highlighted code is the prompt a.k.a. the text proceeding SStuB statement that Codex uses for completion. The purple highlighted code is the code after the SStuB.
  • Figure 2: This annotation tool helps mark Codex completions that do not match any SStuB directly. This guarantees our evaluation is not missing reasonable alternatives to the SStuB that could be deemed a bug or fix.
  • Figure 3: Match rate of Codex Davinci (left). Completions that do not match a patch or SStuB are validated by hand (right).
  • Figure 4: Adding neural-generated comments, step by step, in the prompt preceeding the SStuB. The first added comment induces greatest improvement in generated code.
  • Figure 5: Developers take more time, measured in commits, to resolve SStuBs that Codex generates. All differences are pairwise statistically significant to p $\leq$ 0.0001.
  • ...and 4 more figures