LLMs in Web Development: Evaluating LLM-Generated PHP Code Unveiling Vulnerabilities and Limitations

TL;DR

This work evaluates the security of PHP code generated by GPT-4-0125-preview at scale by building ChatPHP-DB, a dataset of 2,500 AI-generated PHP programs with associated init.sql files deployed in Docker. Using Burp Suite dynamic scans, static analysis, manual code audits, and penetration testing, the study identifies widespread vulnerabilities across Insecure File Upload, SQL Injection, and both Stored and Reflected XSS, with notable rates of exploitable weaknesses ($11.16\%$, $78\%$, and $54.28\%$ in respective analyses). The results demonstrate that while automated scanners can uncover many issues, significant limitations and false negatives/positives persist, especially given the simplicity of many samples and the challenges of testing complex web interactions. The findings underscore the importance of thorough testing and human review when integrating generative AI into web development, and they provide a publicly available resource for ongoing research into secure AI-generated code.

Abstract

This study evaluates the security of web application code generated by Large Language Models, analyzing 2,500 GPT-4 generated PHP websites. These were deployed in Docker containers and tested for vulnerabilities using a hybrid approach of Burp Suite active scanning, static analysis, and manual review. Our investigation focuses on identifying Insecure File Upload, SQL Injection, Stored XSS, and Reflected XSS in GPT-4 generated PHP code. This analysis highlights potential security risks and the implications of deploying such code in real-world scenarios. Overall, our analysis found 2,440 vulnerable parameters. According to Burp's Scan, 11.56% of the sites can be straight out compromised. Adding static scan results, 26% had at least one vulnerability that can be exploited through web interaction. Certain coding scenarios, like file upload functionality, are insecure 78% of the time, underscoring significant risks to software safety and security. To support further research, we have made the source codes and a detailed vulnerability record for each sample publicly available. This study emphasizes the crucial need for thorough testing and evaluation if generative AI technologies are used in software development.

Figures (3)

• Figure 1: Vulnerable code completion example
• Figure 2: Dataset Generation and Vulnerability Labelling Methodology
• Figure 3: Dynamic code generation prompt