Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NOIR: Privacy-Preserving Generation of Code with Open-Source LLMs

Khoa Nguyen, Khiem Ton, NhatHai Phan, Issa Khalil, Khang Tran, Cristian Borcea, Ruoming Jin, Abdallah Khreishah, My T. Thai

TL;DR

NOIR tackles private code generation by partitioning an open-source LLM and equipping the client with INDVocab and LTokenizer to protect prompts and generated code from cloud observation. It combines STuning for efficient client-side fine-tuning and a split-learning design that keeps the encoder/decoder on the client while the cloud handles the middle blocks, enabling indistinguishability at the token-embedding level while preserving code functionality. Extensive experiments against reconstruction attacks show NOIR substantially reduces leakage and outperforms state-of-the-art privacy baselines on standard code-generation benchmarks, with only modest utility loss and significant client-side cost reductions. The work demonstrates practical privacy-preserving code generation using open-source LLMs and provides an open-source release and API to encourage real-world adoption.

Abstract

Although boosting software development performance, large language model (LLM)-powered code generation introduces intellectual property and data security risks rooted in the fact that a service provider (cloud) observes a client's prompts and generated code, which can be proprietary in commercial systems. To mitigate this problem, we propose NOIR, the first framework to protect the client's prompts and generated code from the cloud. NOIR uses an encoder and a decoder at the client to encode and send the prompts' embeddings to the cloud to get enriched embeddings from the LLM, which are then decoded to generate the code locally at the client. Since the cloud can use the embeddings to infer the prompt and the generated code, NOIR introduces a new mechanism to achieve indistinguishability, a local differential privacy protection at the token embedding level, in the vocabulary used in the prompts and code, and a data-independent and randomized tokenizer on the client side. These components effectively defend against reconstruction and frequency analysis attacks by an honest-but-curious cloud. Extensive analysis and results using open-source LLMs show that NOIR significantly outperforms existing baselines on benchmarks, including the Evalplus (MBPP and HumanEval, Pass@1 of 76.7 and 77.4), and BigCodeBench (Pass@1 of 38.7, only a 1.77% drop from the original LLM) under strong privacy against attacks.

NOIR: Privacy-Preserving Generation of Code with Open-Source LLMs

TL;DR

NOIR tackles private code generation by partitioning an open-source LLM and equipping the client with INDVocab and LTokenizer to protect prompts and generated code from cloud observation. It combines STuning for efficient client-side fine-tuning and a split-learning design that keeps the encoder/decoder on the client while the cloud handles the middle blocks, enabling indistinguishability at the token-embedding level while preserving code functionality. Extensive experiments against reconstruction attacks show NOIR substantially reduces leakage and outperforms state-of-the-art privacy baselines on standard code-generation benchmarks, with only modest utility loss and significant client-side cost reductions. The work demonstrates practical privacy-preserving code generation using open-source LLMs and provides an open-source release and API to encourage real-world adoption.

Abstract

Although boosting software development performance, large language model (LLM)-powered code generation introduces intellectual property and data security risks rooted in the fact that a service provider (cloud) observes a client's prompts and generated code, which can be proprietary in commercial systems. To mitigate this problem, we propose NOIR, the first framework to protect the client's prompts and generated code from the cloud. NOIR uses an encoder and a decoder at the client to encode and send the prompts' embeddings to the cloud to get enriched embeddings from the LLM, which are then decoded to generate the code locally at the client. Since the cloud can use the embeddings to infer the prompt and the generated code, NOIR introduces a new mechanism to achieve indistinguishability, a local differential privacy protection at the token embedding level, in the vocabulary used in the prompts and code, and a data-independent and randomized tokenizer on the client side. These components effectively defend against reconstruction and frequency analysis attacks by an honest-but-curious cloud. Extensive analysis and results using open-source LLMs show that NOIR significantly outperforms existing baselines on benchmarks, including the Evalplus (MBPP and HumanEval, Pass@1 of 76.7 and 77.4), and BigCodeBench (Pass@1 of 38.7, only a 1.77% drop from the original LLM) under strong privacy against attacks.
Paper Structure (25 sections, 5 theorems, 25 equations, 19 figures, 3 tables, 1 algorithm)

This paper contains 25 sections, 5 theorems, 25 equations, 19 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. NOIR: Overview, Feasibility, and Threats
  5. Threat Model
  6. NOIR's Defense
  7. Reconstruction Attacks
  8. Prompt Reconstruction Attack
  9. Code Reconstruction Attack
  10. Token-level Privacy Metrics
  11. Initial Assessment of PromptRA, CodeRA
  12. IND-preserving Vocabulary
  13. INDVocab
  14. Prompt-Level Protection
  15. Advantages of INDVocab
  16. ...and 10 more sections

Key Result

Theorem 1

Randomizing the $i^\text{th}$-feature in a token embedding $e_t$ preserves $\epsilon_i$-IND if $\varepsilon_i$ and $\beta_i$ are bounded as where $\Delta^i_{t, min} = \min \{\Delta^i_{t, k}\}_{k \in V \setminus t}$, $\Delta^i_{t, max} = \max \{\Delta^i_{t, k}\}_{k \in V \setminus t}$.

Figures (19)

  • Figure 1: NOIR: Privacy-preserving Generation of Code.
  • Figure 2: Threat Model of an Honest-but-Curious Cloud.
  • Figure 3: An Example of the Threat Model.
  • Figure 4: (a) the percentage of tokens changes in a prompt, (b) the $L_1$-norm distance between original and IND-preserving token embeddings on the MBPP dataset, $\epsilon = 13$.
  • Figure 5: NOIR's gain in terms of (a) bi-gram angular change in every prompt and (b) angular change among tokens in the vocabulary compared with T2T and SnD.
  • ...and 14 more figures

Theorems & Definitions (12)

  • Definition 1
  • Definition 2
  • Theorem 1
  • proof
  • Theorem 2
  • proof
  • Theorem 3
  • proof
  • Proposition 1
  • proof
  • ...and 2 more