Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

WildCode: An Empirical Analysis of Code Generated by ChatGPT

Kobra Khanmohammadi, Pooria Roy, Raphael Khoury, Abdelwahab Hamou-Lhadj, Wilfried Patrick Konan

TL;DR

This paper presents the first large-scale empirical study of code generated by ChatGPT using real-world interactions from the WildChat dataset. It combines syntax validation, OpenGrep-based security analysis, and user-intent classification to characterize both code quality and how users engage with AI-generated code. The findings reveal pervasive security vulnerabilities, notable memory- and deserialization issues, frequent ReDoS risk in regexes, and a substantial presence of hallucinated modules, alongside a clear gap in user attention to secure coding. The work underscores the need for security-aware LLMs, better prompting strategies, and proactive safeguards in AI-assisted programming, and provides a reproducible dataset pipeline for future research.

Abstract

LLM models are increasingly used to generate code, but the quality and security of this code are often uncertain. Several recent studies have raised alarm bells, indicating that such AI-generated code may be particularly vulnerable to cyberattacks. However, most of these studies rely on code that is generated specifically for the study, which raises questions about the realism of such experiments. In this study, we perform a large-scale empirical analysis of real-life code generated by ChatGPT. We evaluate code generated by ChatGPT both with respect to correctness and security and delve into the intentions of users who request code from the model. Our research confirms previous studies that used synthetic queries and yielded evidence that LLM-generated code is often inadequate with respect to security. We also find that users exhibit little curiosity about the security features of the code they ask LLMs to generate, as evidenced by their lack of queries on this topic.

WildCode: An Empirical Analysis of Code Generated by ChatGPT

TL;DR

This paper presents the first large-scale empirical study of code generated by ChatGPT using real-world interactions from the WildChat dataset. It combines syntax validation, OpenGrep-based security analysis, and user-intent classification to characterize both code quality and how users engage with AI-generated code. The findings reveal pervasive security vulnerabilities, notable memory- and deserialization issues, frequent ReDoS risk in regexes, and a substantial presence of hallucinated modules, alongside a clear gap in user attention to secure coding. The work underscores the need for security-aware LLMs, better prompting strategies, and proactive safeguards in AI-assisted programming, and provides a reproducible dataset pipeline for future research.

Abstract

LLM models are increasingly used to generate code, but the quality and security of this code are often uncertain. Several recent studies have raised alarm bells, indicating that such AI-generated code may be particularly vulnerable to cyberattacks. However, most of these studies rely on code that is generated specifically for the study, which raises questions about the realism of such experiments. In this study, we perform a large-scale empirical analysis of real-life code generated by ChatGPT. We evaluate code generated by ChatGPT both with respect to correctness and security and delve into the intentions of users who request code from the model. Our research confirms previous studies that used synthetic queries and yielded evidence that LLM-generated code is often inadequate with respect to security. We also find that users exhibit little curiosity about the security features of the code they ask LLMs to generate, as evidenced by their lack of queries on this topic.

Paper Structure

This paper contains 22 sections, 6 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Review of the Literature
  3. Construction of the Dataset
  4. Code Analysis
  5. Overview of the Code Snippets
  6. Syntax Check
  7. Security Analysis
  8. Weak Cryptographic hash functions:
  9. SQL Injection:
  10. Weak random number generation:
  11. Deserialization attacks:
  12. Memory Safety:
  13. ReDoS (Regular Expression Denial of Service)
  14. Hallucinations
  15. User Intent
  16. ...and 7 more sections

Figures (6)

  • Figure 1: Dataset generation pipeline
  • Figure 2: Hallucination detection pipeline
  • Figure 3: Comparison of programming languages in initial code generation vs. user-requested follow-up.
  • Figure 4: Predicted category distribution for user queries.
  • Figure 5: Distribution of conversation length grouped by initial and followup message category.
  • ...and 1 more figures