Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Prompting Techniques for Secure Code Generation: A Systematic Investigation

Catherine Tony, Nicolás E. Díaz Ferreyra, Markus Mutas, Salem Dhiff, Riccardo Scandariato

TL;DR

This study systematically catalogs prompting techniques for NL-to-code generation and empirically evaluates their impact on security across GPT-3, GPT-3.5, and GPT-4 using the LLMSecEval dataset. It identifies a 5-category taxonomy of prompting strategies, and demonstrates that refinement-based prompts, especially Recursive Criticism and Improvement (RCI), substantially reduce security weaknesses in the generated code, with GPT-4 showing the largest gains. The results generalize to C language to an extent, though effectiveness of techniques varies by language, and data-leakage risks are found to be minimal under API usage. The work provides actionable prompt templates and a replication package to guide practitioners toward more secure LLM-generated code in real-world development pipelines.

Abstract

Large Language Models (LLMs) are gaining momentum in software development with prompt-driven programming enabling developers to create code from natural language (NL) instructions. However, studies have questioned their ability to produce secure code and, thereby, the quality of prompt-generated software. Alongside, various prompting techniques that carefully tailor prompts have emerged to elicit optimal responses from LLMs. Still, the interplay between such prompting strategies and secure code generation remains under-explored and calls for further investigations. OBJECTIVE: In this study, we investigate the impact of different prompting techniques on the security of code generated from NL instructions by LLMs. METHOD: First we perform a systematic literature review to identify the existing prompting techniques that can be used for code generation tasks. A subset of these techniques are evaluated on GPT-3, GPT-3.5, and GPT-4 models for secure code generation. For this, we used an existing dataset consisting of 150 NL security-relevant code-generation prompts. RESULTS: Our work (i) classifies potential prompting techniques for code generation (ii) adapts and evaluates a subset of the identified techniques for secure code generation tasks and (iii) observes a reduction in security weaknesses across the tested LLMs, especially after using an existing technique called Recursive Criticism and Improvement (RCI), contributing valuable insights to the ongoing discourse on LLM-generated code security.

Prompting Techniques for Secure Code Generation: A Systematic Investigation

TL;DR

This study systematically catalogs prompting techniques for NL-to-code generation and empirically evaluates their impact on security across GPT-3, GPT-3.5, and GPT-4 using the LLMSecEval dataset. It identifies a 5-category taxonomy of prompting strategies, and demonstrates that refinement-based prompts, especially Recursive Criticism and Improvement (RCI), substantially reduce security weaknesses in the generated code, with GPT-4 showing the largest gains. The results generalize to C language to an extent, though effectiveness of techniques varies by language, and data-leakage risks are found to be minimal under API usage. The work provides actionable prompt templates and a replication package to guide practitioners toward more secure LLM-generated code in real-world development pipelines.

Abstract

Large Language Models (LLMs) are gaining momentum in software development with prompt-driven programming enabling developers to create code from natural language (NL) instructions. However, studies have questioned their ability to produce secure code and, thereby, the quality of prompt-generated software. Alongside, various prompting techniques that carefully tailor prompts have emerged to elicit optimal responses from LLMs. Still, the interplay between such prompting strategies and secure code generation remains under-explored and calls for further investigations. OBJECTIVE: In this study, we investigate the impact of different prompting techniques on the security of code generated from NL instructions by LLMs. METHOD: First we perform a systematic literature review to identify the existing prompting techniques that can be used for code generation tasks. A subset of these techniques are evaluated on GPT-3, GPT-3.5, and GPT-4 models for secure code generation. For this, we used an existing dataset consisting of 150 NL security-relevant code-generation prompts. RESULTS: Our work (i) classifies potential prompting techniques for code generation (ii) adapts and evaluates a subset of the identified techniques for secure code generation tasks and (iii) observes a reduction in security weaknesses across the tested LLMs, especially after using an existing technique called Recursive Criticism and Improvement (RCI), contributing valuable insights to the ongoing discourse on LLM-generated code security.
Paper Structure (52 sections, 13 figures, 7 tables)

This paper contains 52 sections, 13 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Code Generation Using LLMs
  4. Security in LLM-Generated Code
  5. Methodology for Systematic Literature Review
  6. Prompting Techniques for Code Generation (RQ1)
  7. Overview of the Selected Papers
  8. Classification of Prompting Techniques
  9. Root Techniques
  10. Refinement-based Techniques
  11. Decomposition-based Techniques
  12. Reasoning-based Techniques
  13. Priming Techniques
  14. Security Evaluation of Prompting Techniques: Methodology
  15. Dataset and Models
  16. ...and 37 more sections

Figures (13)

  • Figure 1: Steps followed for the SLR on prompting techniques that are suitable for code generation
  • Figure 2: Classification of prompting techniques for code generation
  • Figure 3: Zero-shot (left) and few-shot (right) prompting with ChatGPT for code generation.
  • Figure 4: RCI (left) and Progressive Hint (right) prompting with ChatGPT for code generation.
  • Figure 5: Feedback generation (left) and refining (right) steps of Self-refining prompting with ChatGPT for code generation.
  • ...and 8 more figures