Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Security-by-Design for LLM-Based Code Generation: Leveraging Internal Representations for Concept-Driven Steering Mechanisms

Maximilian Wendlinger, Daniel Kowatsch, Konstantin Böttinger, Philip Sperl

Abstract

Large Language Models (LLMs) show remarkable capabilities in understanding natural language and generating complex code. However, as practitioners adopt CodeLLMs for increasingly critical development tasks, research reveals that these models frequently generate functionally correct yet insecure code, posing significant security risks. While multiple approaches have been proposed to improve security in AI-based code generation, combined benchmarks show these methods remain insufficient for practical use, achieving only limited improvements in both functional correctness and security. This stems from a fundamental gap in understanding the internal mechanisms of code generation and the root causes of security vulnerabilities, forcing researchers to rely on heuristics and empirical observations. In this work, we investigate the internal representation of security concepts in CodeLLMs, revealing that models are often aware of vulnerabilities as they generate insecure code. Through systematic evaluation, we demonstrate that CodeLLMs can distinguish between security subconcepts, enabling a more fine-grained analysis than prior black-box approaches. Leveraging these insights, we propose Secure Concept Steering for CodeLLMs (SCS-Code). During token generation, SCS-Code steers LLMs' internal representations toward secure and functional code output, enabling a lightweight and modular mechanism that can be integrated into existing code models. Our approach achieves superior performance compared to state-of-the-art methods across multiple secure coding benchmarks.

Security-by-Design for LLM-Based Code Generation: Leveraging Internal Representations for Concept-Driven Steering Mechanisms

Abstract

Large Language Models (LLMs) show remarkable capabilities in understanding natural language and generating complex code. However, as practitioners adopt CodeLLMs for increasingly critical development tasks, research reveals that these models frequently generate functionally correct yet insecure code, posing significant security risks. While multiple approaches have been proposed to improve security in AI-based code generation, combined benchmarks show these methods remain insufficient for practical use, achieving only limited improvements in both functional correctness and security. This stems from a fundamental gap in understanding the internal mechanisms of code generation and the root causes of security vulnerabilities, forcing researchers to rely on heuristics and empirical observations. In this work, we investigate the internal representation of security concepts in CodeLLMs, revealing that models are often aware of vulnerabilities as they generate insecure code. Through systematic evaluation, we demonstrate that CodeLLMs can distinguish between security subconcepts, enabling a more fine-grained analysis than prior black-box approaches. Leveraging these insights, we propose Secure Concept Steering for CodeLLMs (SCS-Code). During token generation, SCS-Code steers LLMs' internal representations toward secure and functional code output, enabling a lightweight and modular mechanism that can be integrated into existing code models. Our approach achieves superior performance compared to state-of-the-art methods across multiple secure coding benchmarks.
Paper Structure (34 sections, 9 equations, 9 figures, 13 tables)

This paper contains 34 sections, 9 equations, 9 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Secure Code Generation
  4. Interpretability of LLMs
  5. Theoretical Background
  6. Concept Representations in LLMs
  7. Concept Extraction
  8. Model Steering
  9. Code Security Concept Representation and Model Steering
  10. Code-Related Concepts
  11. Code Security Concept Extraction
  12. Concept Comparison
  13. Code Security Subconcepts
  14. Code Security Steering
  15. Measures of Code Quality
  16. ...and 19 more sections

Figures (9)

  • Figure 1: Overview of the general framework for concept extraction and model steering.
  • Figure 2: Overview of different analysis dimensions for the code security (sub-) concepts.
  • Figure 3: Residual stream activations for different layers of Llama3.1-8B projected onto the second principal component, corresponding to the contrastive samples in the Python code security dataset.
  • Figure 4: Alignment of residual stream activations and code security concept vector for Python code, where we color the three highest and lowest alignment values in blue and red shades, respectively.
  • Figure 5: PCA and t-SNE projections of concept vectors for a range of concepts from panickssery2024, and code security concepts for different programming languages.
  • ...and 4 more figures