Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Deep Dive Into Large Language Model Code Generation Mistakes: What and Why?

QiHong Chen, Jiachen Yu, Jiawei Li, Jiecheng Deng, Justin Tian Jin Chen, Iftekhar Ahmed

TL;DR

The paper investigates non-syntactic mistakes in LLM-generated code across Python and Java, revealing 17 mistake types within 7 high-level categories and 6 underlying reasons. It leverages two large multilingual datasets and Automated Program Repair to ground incorrect code against correct behavior, plus a semi-automated verification of causes. The study demonstrates that GPT-4 with ReAct prompting can identify mistake causes with competitive precision (up to 0.97) and a notably improved reason-identification F1 (up to 0.78) compared to baselines, highlighting the value of structured prompting and tooling. These findings offer actionable guidance for designing prompts, demonstrations, and verification methods to improve reliability and safety of LLM-generated code in real-world software development.

Abstract

Recent advancements in Large Language Models (LLMs) have led to their widespread application in automated code generation. However, these models can still generate defective code that deviates from the specification. Previous research has mainly focused on the mistakes in LLM-generated standalone functions, overlooking real-world software development situations where the successful generation of the code requires software contexts such as external dependencies. In this paper, we considered both of these code generation situations and identified a range of \textit{non-syntactic mistakes} arising from LLMs' misunderstandings of coding question specifications. Seven categories of non-syntactic mistakes were identified through extensive manual analyses, four of which were missed by previous works. To better understand these mistakes, we proposed six reasons behind these mistakes from various perspectives. Moreover, we explored the effectiveness of LLMs in detecting mistakes and their reasons. Our evaluation demonstrated that GPT-4 with the ReAct prompting technique can achieve an F1 score of up to 0.65 when identifying reasons for LLM's mistakes, such as misleading function signatures. We believe that these findings offer valuable insights into enhancing the quality of LLM-generated code.

A Deep Dive Into Large Language Model Code Generation Mistakes: What and Why?

TL;DR

The paper investigates non-syntactic mistakes in LLM-generated code across Python and Java, revealing 17 mistake types within 7 high-level categories and 6 underlying reasons. It leverages two large multilingual datasets and Automated Program Repair to ground incorrect code against correct behavior, plus a semi-automated verification of causes. The study demonstrates that GPT-4 with ReAct prompting can identify mistake causes with competitive precision (up to 0.97) and a notably improved reason-identification F1 (up to 0.78) compared to baselines, highlighting the value of structured prompting and tooling. These findings offer actionable guidance for designing prompts, demonstrations, and verification methods to improve reliability and safety of LLM-generated code in real-world software development.

Abstract

Recent advancements in Large Language Models (LLMs) have led to their widespread application in automated code generation. However, these models can still generate defective code that deviates from the specification. Previous research has mainly focused on the mistakes in LLM-generated standalone functions, overlooking real-world software development situations where the successful generation of the code requires software contexts such as external dependencies. In this paper, we considered both of these code generation situations and identified a range of \textit{non-syntactic mistakes} arising from LLMs' misunderstandings of coding question specifications. Seven categories of non-syntactic mistakes were identified through extensive manual analyses, four of which were missed by previous works. To better understand these mistakes, we proposed six reasons behind these mistakes from various perspectives. Moreover, we explored the effectiveness of LLMs in detecting mistakes and their reasons. Our evaluation demonstrated that GPT-4 with the ReAct prompting technique can achieve an F1 score of up to 0.65 when identifying reasons for LLM's mistakes, such as misleading function signatures. We believe that these findings offer valuable insights into enhancing the quality of LLM-generated code.

Paper Structure

This paper contains 24 sections, 1 equation, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. LLMs for Code Generation
  4. Mistakes in LLM Generated Code
  5. Why LLMs Make Mistakes in Code Generation
  6. Experimental Setup
  7. RQ1:What mistakes do LLMs make in the code generation?
  8. Methodology to Answer RQ1
  9. Answer to RQ1: Non-Syntactic Mistake Analysis
  10. Mistake Severity Categorization
  11. Analysis of Mistake Types and Severity
  12. RQ2: What are the underlying reasons for the LLM's mistakes in the code generation task?
  13. Methodology to Answer RQ2
  14. Reason Identification
  15. Answer to RQ2: Reasons of Non-syntactic Mistakes
  16. ...and 9 more sections

Figures (3)

  • Figure 1: LLM Code Generation Mistakes Examples-IUE, IMKE, ARHE, IDS
  • Figure 2: LLM Code Generation Mistakes Examples-IRE, CME
  • Figure 3: Example Reasons for LLM Code Generation Mistakes