Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Large Language Models for Code: Security Hardening and Adversarial Testing

Jingxuan He, Martin Vechev

TL;DR

The paper tackles the security of large language models used for code by introducing controlled code generation, which guides outputs toward secure or unsafe outcomes via a binary property while preserving functional correctness. It presents SVEN, a modular prefix-tuning approach that attaches small, property-specific prefixes to frozen LMs, combined with region-aware losses and a carefully curated training dataset derived from real commits. SVEN achieves strong security control (e.g., CodeGen-2.7B: secure rate rising from 59.1% to 92.3% with SVENsec) while maintaining HumanEval performance close to the unmodified LM, and provides insight into generalizability across models, CWEs, and prompt perturbations. The work offers practical implications for hardening code-completion tools and for rigorous adversarial evaluation, while acknowledging data quality challenges and avenues for broader CWE/language coverage.

Abstract

Large language models (large LMs) are increasingly trained on massive codebases and used to generate code. However, LMs lack awareness of security and are found to frequently produce unsafe code. This work studies the security of LMs along two important axes: (i) security hardening, which aims to enhance LMs' reliability in generating secure code, and (ii) adversarial testing, which seeks to evaluate LMs' security at an adversarial standpoint. We address both of these by formulating a new security task called controlled code generation. The task is parametric and takes as input a binary property to guide the LM to generate secure or unsafe code, while preserving the LM's capability of generating functionally correct code. We propose a novel learning-based approach called SVEN to solve this task. SVEN leverages property-specific continuous vectors to guide program generation towards the given property, without modifying the LM's weights. Our training procedure optimizes these continuous vectors by enforcing specialized loss terms on different regions of code, using a high-quality dataset carefully curated by us. Our extensive evaluation shows that SVEN is highly effective in achieving strong security control. For instance, a state-of-the-art CodeGen LM with 2.7B parameters generates secure code for 59.1% of the time. When we employ SVEN to perform security hardening (or adversarial testing) on this LM, the ratio is significantly boosted to 92.3% (or degraded to 36.8%). Importantly, SVEN closely matches the original LMs in functional correctness.

Large Language Models for Code: Security Hardening and Adversarial Testing

TL;DR

The paper tackles the security of large language models used for code by introducing controlled code generation, which guides outputs toward secure or unsafe outcomes via a binary property while preserving functional correctness. It presents SVEN, a modular prefix-tuning approach that attaches small, property-specific prefixes to frozen LMs, combined with region-aware losses and a carefully curated training dataset derived from real commits. SVEN achieves strong security control (e.g., CodeGen-2.7B: secure rate rising from 59.1% to 92.3% with SVENsec) while maintaining HumanEval performance close to the unmodified LM, and provides insight into generalizability across models, CWEs, and prompt perturbations. The work offers practical implications for hardening code-completion tools and for rigorous adversarial evaluation, while acknowledging data quality challenges and avenues for broader CWE/language coverage.

Abstract

Large language models (large LMs) are increasingly trained on massive codebases and used to generate code. However, LMs lack awareness of security and are found to frequently produce unsafe code. This work studies the security of LMs along two important axes: (i) security hardening, which aims to enhance LMs' reliability in generating secure code, and (ii) adversarial testing, which seeks to evaluate LMs' security at an adversarial standpoint. We address both of these by formulating a new security task called controlled code generation. The task is parametric and takes as input a binary property to guide the LM to generate secure or unsafe code, while preserving the LM's capability of generating functionally correct code. We propose a novel learning-based approach called SVEN to solve this task. SVEN leverages property-specific continuous vectors to guide program generation towards the given property, without modifying the LM's weights. Our training procedure optimizes these continuous vectors by enforcing specialized loss terms on different regions of code, using a high-quality dataset carefully curated by us. Our extensive evaluation shows that SVEN is highly effective in achieving strong security control. For instance, a state-of-the-art CodeGen LM with 2.7B parameters generates secure code for 59.1% of the time. When we employ SVEN to perform security hardening (or adversarial testing) on this LM, the ratio is significantly boosted to 92.3% (or degraded to 36.8%). Importantly, SVEN closely matches the original LMs in functional correctness.
Paper Structure (78 sections, 8 equations, 22 figures, 7 tables)

This paper contains 78 sections, 8 equations, 22 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Security Hardening and Adversarial Testing
  3. Challenge I: Modularity
  4. Challenge II: Functional Correctness vs. Security Control
  5. Challenge III: Ensuring High-quality Training Data
  6. Our Solution: SVEN
  7. Evaluating SVEN
  8. SVEN's Security Implications
  9. Main Contributions
  10. Background and Related Work
  11. Code Generation with Large Language Models
  12. LMs' Benefits in Programming Productivity
  13. Code Security and Vulnerability
  14. Security of LMs for Code
  15. Controlled Code Generation
  16. ...and 63 more sections

Figures (22)

  • Figure 1: A conceptual visualization of our objective for security hardening and adversarial testing.
  • Figure 2: Visualization of controlled code generation vs. vulnerability detection, repair, and injection.
  • Figure 3: A Python function before and after a cross-site scripting vulnerability gets fixed in a GitHub commit*.
  • Figure 3: Comparison between CodeGen LMs DBLP:journals/corr/abs-2203-13474 and SVEN on the ability to generate functionally correct code, measured by pass@$k$ scores on the HumanEval benchmark DBLP:journals/corr/abs-2107-03374.
  • Figure 4: Inference procedures of (\ref{['fig:inference-lm']}) LM and (\ref{['fig:inference-tool']}) SVENsec.
  • ...and 17 more figures