Understanding Privacy Risks in Code Models Through Training Dynamics: A Causal Approach

TL;DR

The paper investigates privacy risks in Large Language Models for Code (LLM4Code) by differentiating PII types and linking their learnability during training to leakage via a causal framework. It constructs a multi-type PII dataset from real-world code, analyzes training dynamics, and applies a structural causal model to estimate the causal effect of learnability on leakage. Key findings show that leakage risk varies by PII type, with IP addresses leaking more for easy-to-learn tokens while keys and passwords leak less, and that some types exhibit mixed or model-dependent behavior; the study provides causal evidence and actionable guidance for type- and learnability-aware defenses. The work highlights the need for refined PII detection and defense strategies, including synthetic data injection, to mitigate leakage while preserving utility in code-generation tasks, and it releases artifacts to support reproducibility.

Abstract

Large language models for code (LLM4Code) have greatly improved developer productivity but also raise privacy concerns due to their reliance on open-source repositories containing abundant personally identifiable information (PII). Prior work shows that commercial models can reproduce sensitive PII, yet existing studies largely treat PII as a single category and overlook the heterogeneous risks among different types. We investigate whether distinct PII types vary in their likelihood of being learned and leaked by LLM4Code, and whether this relationship is causal. Our methodology includes building a dataset with diverse PII types, fine-tuning representative models of different scales, computing training dynamics on real PII data, and formulating a structural causal model to estimate the causal effect of learnability on leakage. Results show that leakage risks differ substantially across PII types and correlate with their training dynamics: easy-to-learn instances such as IP addresses exhibit higher leakage, while harder types such as keys and passwords leak less frequently. Ambiguous types show mixed behaviors. This work provides the first causal evidence that leakage risks are type-dependent and offers guidance for developing type-aware and learnability-aware defenses for LLM4Code.

Figures (8)

• Figure 1: Overview of our study workflow. The process starts with PII dataset construction, followed by collection of training dynamics during fine-tuning, execution of the PII attack, and finally causal analysis of the link between training dynamics and attack success rate.
• Figure 2: Examples where real PII values are leaked when the LLM4Code is queried with surrounding context. Leaked values are redacted with black boxes and query inputs are highlighted in yellow.
• Figure 3: Dataset Curation Process
• Figure 4: Structural causal model for the relationship between learning dynamics and PII leakage.
• Figure 5: Training loss curves of different LLM4Code models across 10 epochs. The abscissa length of the curves varies because of the different batch sizes of the models.