Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

How Safe Are AI-Generated Patches? A Large-scale Study on Security Risks in LLM and Agentic Automated Program Repair on SWE-bench

Amirali Sajadi, Kostadin Damevski, Preetha Chatterjee

TL;DR

This work delivers the first large-scale security analysis of patches produced by LLMs and agentic APR systems on real-world SWE-bench issues, comparing them to developer-written fixes. Using SWE-bench, it shows that standalone Llama 3.3 patches introduce markedly more vulnerabilities than developers, with distinct CWE patterns such as eval injection and insecure deserialization. Agentic APR frameworks also generate vulnerabilities, especially when given broad autonomy that leads to extensive cross-file edits. The study further links vulnerability likelihood to code- and issue-level factors (e.g., number of files touched, information completeness in issues) rather than project-scale metrics, and proposes actionable guidance for risk assessment and safer deployment of AI-assisted repair tools.

Abstract

Large language models (LLMs) and their agentic frameworks are increasingly adopted to perform development tasks such as automated program repair (APR). While prior work has identified security risks in LLM-generated code, most have focused on synthetic, simplified, or isolated tasks that lack the complexity of real-world program repair. In this study, we present the first large-scale security analysis of LLM-generated patches using 20,000+ GitHub issues. We evaluate patches proposed by developers, a standalone LLM (Llama 3.3 Instruct-70B), and three top-performing agentic frameworks (OpenHands, AutoCodeRover, HoneyComb). Finally, we analyze a wide range of code, issue, and project-level factors to understand the conditions under which generating insecure patches is more likely. Our findings reveal that Llama introduces many new vulnerabilities, exhibiting unique patterns not found in developers' code. Agentic workflows also generate a number of vulnerabilities, particularly when given more autonomy. We find that vulnerabilities in LLM-generated patches are associated with distinctive code characteristics and are commonly observed in issues missing specific types of information. These results suggest that contextual factors play a critical role in the security of the generated patches and point toward the need for proactive risk assessment methods that account for both issue and code-level information.

How Safe Are AI-Generated Patches? A Large-scale Study on Security Risks in LLM and Agentic Automated Program Repair on SWE-bench

TL;DR

This work delivers the first large-scale security analysis of patches produced by LLMs and agentic APR systems on real-world SWE-bench issues, comparing them to developer-written fixes. Using SWE-bench, it shows that standalone Llama 3.3 patches introduce markedly more vulnerabilities than developers, with distinct CWE patterns such as eval injection and insecure deserialization. Agentic APR frameworks also generate vulnerabilities, especially when given broad autonomy that leads to extensive cross-file edits. The study further links vulnerability likelihood to code- and issue-level factors (e.g., number of files touched, information completeness in issues) rather than project-scale metrics, and proposes actionable guidance for risk assessment and safer deployment of AI-assisted repair tools.

Abstract

Large language models (LLMs) and their agentic frameworks are increasingly adopted to perform development tasks such as automated program repair (APR). While prior work has identified security risks in LLM-generated code, most have focused on synthetic, simplified, or isolated tasks that lack the complexity of real-world program repair. In this study, we present the first large-scale security analysis of LLM-generated patches using 20,000+ GitHub issues. We evaluate patches proposed by developers, a standalone LLM (Llama 3.3 Instruct-70B), and three top-performing agentic frameworks (OpenHands, AutoCodeRover, HoneyComb). Finally, we analyze a wide range of code, issue, and project-level factors to understand the conditions under which generating insecure patches is more likely. Our findings reveal that Llama introduces many new vulnerabilities, exhibiting unique patterns not found in developers' code. Agentic workflows also generate a number of vulnerabilities, particularly when given more autonomy. We find that vulnerabilities in LLM-generated patches are associated with distinctive code characteristics and are commonly observed in issues missing specific types of information. These results suggest that contextual factors play a critical role in the security of the generated patches and point toward the need for proactive risk assessment methods that account for both issue and code-level information.

Paper Structure

This paper contains 25 sections, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Dataset and Task Setup
  4. Patch Generation with LLM and Agentic Frameworks
  5. RQ1. Llama 3.3 Instruct (70B)
  6. RQ2. Agentic LLM Frameworks
  7. Developer-written Patch Collection
  8. Vulnerability Detection
  9. Metrics for Empirical Analysis
  10. RQ1 & RQ2: Vulnerability Prevalence and Properties
  11. RQ3: Code, Issue, Project Characteristics Associated with LLM or agent-generated Vulnerabilities
  12. Results
  13. RQ1: Security of patches generated by Developers vs. Llama 3.3
  14. RQ2: Security of patches generated by the agentic APR workflows
  15. RQ3: What code, issue, or project characteristics are associated with the generation of vulnerable code by LLMs when resolving issues?
  16. ...and 10 more sections

Figures (2)

  • Figure 1: Overview of Our Methodology for LLM-Based Code Generation and Security Evaluation.
  • Figure 2: Number of changed files per instance (capped at 10). Vulnerable cases show greater variability and significantly higher averages than the general set.