Is Your Prompt Poisoning Code? Defect Induction Rates and Security Mitigation Strategies

TL;DR

This work addresses how benign but poorly specified prompts degrade the security of AI-generated code. It introduces a three-dimensional prompt-normativity framework and CWE-BENCH-PYTHON, a large-scale benchmark spanning $L_0$–$L_3$ prompts across 8 pillar and 33 base CWEs, to quantify vulnerability rates via the metric $\text{func-sec}@k(L_x)=1-\frac{c_{L_x}}{n_{L_x}}$ with a 0.9 confidence threshold. The study finds a robust, cross-model negative correlation between prompt clarity/completeness/logical consistency and code security, especially for logic-intensive weaknesses, and shows that Chain-of-Thought and Self-Correction can mitigate these risks. Practically, the results imply that improving prompt quality is an effective, actionable defense for securing AI-assisted software development, and they highlight avenues for model-internal safeguards and future benchmark expansion.

Abstract

Large language models (LLMs) have become indispensable for automated code generation, yet the quality and security of their outputs remain a critical concern. Existing studies predominantly concentrate on adversarial attacks or inherent flaws within the models. However, a more prevalent yet underexplored issue concerns how the quality of a benign but poorly formulated prompt affects the security of the generated code. To investigate this, we first propose an evaluation framework for prompt quality encompassing three key dimensions: goal clarity, information completeness, and logical consistency. Based on this framework, we construct and publicly release CWE-BENCH-PYTHON, a large-scale benchmark dataset containing tasks with prompts categorized into four distinct levels of normativity (L0-L3). Extensive experiments on multiple state-of-the-art LLMs reveal a clear correlation: as prompt normativity decreases, the likelihood of generating insecure code consistently and markedly increases. Furthermore, we demonstrate that advanced prompting techniques, such as Chain-of-Thought and Self-Correction, effectively mitigate the security risks introduced by low-quality prompts, substantially improving code safety. Our findings highlight that enhancing the quality of user prompts constitutes a critical and effective strategy for strengthening the security of AI-generated code.

Figures (9)

• Figure 1: Four Levels of Specification in Task Design
• Figure 2: CWE Vulnerabilities Classification
• Figure 3: Security Code Review and Vulnerability Detection for LLM-Generated Code Based on CWE Tasks. Each CWE Scenario Includes Specific Evaluation Templates for Its Five Associated Tasks.
• Figure 4: Workflow for Code Generation Security Dataset Creation and Evaluation
• Figure 5: Trend of Average Performance for Pillar CWEs across L0-L3 in Code Generation by Different Models. The green line above the bars indicates the strong correlation between code security and prompt specification.