Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Protecting Private Code in IDE Autocomplete using Differential Privacy

Evgeny Grigorenko, David Stanojević, David Ilić, Egor Bogomolov, Kostadin Cvejoski

TL;DR

The paper tackles privacy risks in LLM-powered IDE code completion by applying formal differential privacy to fine-tune a code-focused Mellum model with LoRA on private Kotlin data. Using DP-SGD, the authors achieve a substantial reduction in membership inference risk, lowering the attack's $AUC$ from $0.901$ to $0.606$ while maintaining utility close to a non-private baseline, even when training on as little as $80{,}000$ examples. The work demonstrates a practical privacy-utility trade-off, arguing that DP provides strong, actionable protection for private code in AI-assisted IDE features. It represents the first code completion LLM with formal DP guarantees, offering a path toward private and trustworthy developer-tools that leverage LLMs.

Abstract

Modern Integrated Development Environments (IDEs) increasingly leverage Large Language Models (LLMs) to provide advanced features like code autocomplete. While powerful, training these models on user-written code introduces significant privacy risks, making the models themselves a new type of data vulnerability. Malicious actors can exploit this by launching attacks to reconstruct sensitive training data or infer whether a specific code snippet was used for training. This paper investigates the use of Differential Privacy (DP) as a robust defense mechanism for training an LLM for Kotlin code completion. We fine-tune a \texttt{Mellum} model using DP and conduct a comprehensive evaluation of its privacy and utility. Our results demonstrate that DP provides a strong defense against Membership Inference Attacks (MIAs), reducing the attack's success rate close to a random guess (AUC from 0.901 to 0.606). Furthermore, we show that this privacy guarantee comes at a minimal cost to model performance, with the DP-trained model achieving utility scores comparable to its non-private counterpart, even when trained on 100x less data. Our findings suggest that DP is a practical and effective solution for building private and trustworthy AI-powered IDE features.

Protecting Private Code in IDE Autocomplete using Differential Privacy

TL;DR

The paper tackles privacy risks in LLM-powered IDE code completion by applying formal differential privacy to fine-tune a code-focused Mellum model with LoRA on private Kotlin data. Using DP-SGD, the authors achieve a substantial reduction in membership inference risk, lowering the attack's from to while maintaining utility close to a non-private baseline, even when training on as little as examples. The work demonstrates a practical privacy-utility trade-off, arguing that DP provides strong, actionable protection for private code in AI-assisted IDE features. It represents the first code completion LLM with formal DP guarantees, offering a path toward private and trustworthy developer-tools that leverage LLMs.

Abstract

Modern Integrated Development Environments (IDEs) increasingly leverage Large Language Models (LLMs) to provide advanced features like code autocomplete. While powerful, training these models on user-written code introduces significant privacy risks, making the models themselves a new type of data vulnerability. Malicious actors can exploit this by launching attacks to reconstruct sensitive training data or infer whether a specific code snippet was used for training. This paper investigates the use of Differential Privacy (DP) as a robust defense mechanism for training an LLM for Kotlin code completion. We fine-tune a \texttt{Mellum} model using DP and conduct a comprehensive evaluation of its privacy and utility. Our results demonstrate that DP provides a strong defense against Membership Inference Attacks (MIAs), reducing the attack's success rate close to a random guess (AUC from 0.901 to 0.606). Furthermore, we show that this privacy guarantee comes at a minimal cost to model performance, with the DP-trained model achieving utility scores comparable to its non-private counterpart, even when trained on 100x less data. Our findings suggest that DP is a practical and effective solution for building private and trustworthy AI-powered IDE features.
Paper Structure (15 sections, 1 equation, 3 figures, 1 table)

This paper contains 15 sections, 1 equation, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Differential Privacy
  4. Membership Inference Attacks
  5. Background and Methodology
  6. LLMs as Code Completion Models
  7. Threat Model
  8. Defense: Differential Privacy
  9. Experimental Setup
  10. Baseline (Non-DP)
  11. Differentially Private Model
  12. Utility Evaluation
  13. Privacy Evaluation
  14. Results
  15. Conclusion and Future Work

Figures (3)

  • Figure 1: An illustration of a Membership Inference Attack (MIA), where an adversary's goal is to determine if a specific data point (e.g., the one containing the avocado) was part of the original private dataset $D$.
  • Figure 2: The workflow of Differentially Private Stochastic Gradient Descent (DP-SGD). Per-sample gradients are first computed and then clipped. These clipped gradients are averaged, noise is added , and the final result is used to update the model
  • Figure 3: Comparison of DP and non-DP models on training dynamics and privacy. Left: Validation loss and privacy budget ($\epsilon$) vs. data points seen. Right: ROC curve for MIA.