Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Taught by the Flawed: How Dataset Insecurity Breeds Vulnerable AI Code

Catherine Xia, Manar H. Alalfi

TL;DR

AI programming assistants frequently generate insecure code, in part due to training data quality. The authors curate a secure Python dataset by filtering with Bandit and train two CodeT5-based transformers on curated versus unfiltered data, evaluating correctness and security on vulnerability-focused prompts. Results show the secure-trained model achieves fewer security flaws with comparable functional correctness, though overall correctness remains low, highlighting limited but actionable benefits of secure data curation. The study suggests data-quality strategies as a viable path to more reliable AI-assisted coding, while underscoring the need for larger datasets, improved evaluation, and complementary techniques to robustly secure generated code.

Abstract

AI programming assistants have demonstrated a tendency to generate code containing basic security vulnerabilities. While developers are ultimately responsible for validating and reviewing such outputs, improving the inherent quality of these generated code snippets remains essential. A key contributing factor to insecure outputs is the presence of vulnerabilities in the training datasets used to build large language models (LLMs). To address this issue, we propose curating training data to include only code that is free from detectable vulnerabilities. In this study, we constructed a secure dataset by filtering an existing Python corpus using a static analysis tool to retain only vulnerability-free functions. We then trained two transformer-based models: one on the curated dataset and one on the original, unfiltered dataset. The models were evaluated on both the correctness and security of the code they generated in response to natural language function descriptions. Our results show that the model trained on the curated dataset produced outputs with fewer security issues, while maintaining comparable functional correctness. These findings highlight the importance of secure training data in improving the reliability of AI-based programming assistants, though further enhancements to model architecture and evaluation are needed to reinforce these outcomes.

Taught by the Flawed: How Dataset Insecurity Breeds Vulnerable AI Code

TL;DR

AI programming assistants frequently generate insecure code, in part due to training data quality. The authors curate a secure Python dataset by filtering with Bandit and train two CodeT5-based transformers on curated versus unfiltered data, evaluating correctness and security on vulnerability-focused prompts. Results show the secure-trained model achieves fewer security flaws with comparable functional correctness, though overall correctness remains low, highlighting limited but actionable benefits of secure data curation. The study suggests data-quality strategies as a viable path to more reliable AI-assisted coding, while underscoring the need for larger datasets, improved evaluation, and complementary techniques to robustly secure generated code.

Abstract

AI programming assistants have demonstrated a tendency to generate code containing basic security vulnerabilities. While developers are ultimately responsible for validating and reviewing such outputs, improving the inherent quality of these generated code snippets remains essential. A key contributing factor to insecure outputs is the presence of vulnerabilities in the training datasets used to build large language models (LLMs). To address this issue, we propose curating training data to include only code that is free from detectable vulnerabilities. In this study, we constructed a secure dataset by filtering an existing Python corpus using a static analysis tool to retain only vulnerability-free functions. We then trained two transformer-based models: one on the curated dataset and one on the original, unfiltered dataset. The models were evaluated on both the correctness and security of the code they generated in response to natural language function descriptions. Our results show that the model trained on the curated dataset produced outputs with fewer security issues, while maintaining comparable functional correctness. These findings highlight the importance of secure training data in improving the reliability of AI-based programming assistants, though further enhancements to model architecture and evaluation are needed to reinforce these outcomes.

Paper Structure

This paper contains 25 sections, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background
  3. CodeSearchNet
  4. Bandit
  5. Transformers
  6. Literature Review
  7. Experiment Setup
  8. Dataset Preparation
  9. Comment and Docstring Removal
  10. Dataset Construction Pipeline
  11. Transformer Model and Training
  12. Prompt Selection and Evaluation Criteria
  13. Results
  14. Dataset Analysis
  15. Transformer Model Training
  16. ...and 10 more sections

Figures (5)

  • Figure 1: Main characteristics of a transformer Model transformer
  • Figure 2: Pipeline for creating datasets
  • Figure 3: Training loss over 5 epochs for both transformer models
  • Figure 4: Correctness evaluation of outputs from each transformer
  • Figure 5: Security evaluation of outputs from each transformer