Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Your Code Secret Belongs to Me: Neural Code Completion Tools Can Memorize Hard-Coded Credentials

Yizhan Huang, Yichen Li, Weibin Wu, Jianping Zhang, Michael R. Lyu

TL;DR

This work demonstrates that Neural Code Completion Tools (NCCTs) can memorize and occasionally reveal hard-coded credentials from their training data. It introduces the Hard-coded Credential Revealer (HCR), a phase-based framework that constructs prompts from GitHub-sourced secrets, applies regex-based identification, and uses entropy, pattern, and word filters to validate candidates, evaluating six NCCTs across 18 secret types. The results show NCCTs can reproduce exact training data and leak additional secrets, including two valid Stripe test keys, highlighting a substantial privacy risk. The authors discuss mitigations spanning data cleaning, privacy-preserving training, and post-processing, underscoring the need for auditing NCCTs before deployment and prompting further research into robust defenses.

Abstract

Neural Code Completion Tools (NCCTs) have reshaped the field of software engineering, which are built upon the language modeling technique and can accurately suggest contextually relevant code snippets. However, language models may emit the training data verbatim during inference with appropriate prompts. This memorization property raises privacy concerns of NCCTs about hard-coded credential leakage, leading to unauthorized access to applications, systems, or networks. Therefore, to answer whether NCCTs will emit the hard-coded credential, we propose an evaluation tool called Hard-coded Credential Revealer (HCR). HCR constructs test prompts based on GitHub code files with credentials to reveal the memorization phenomenon of NCCTs. Then, HCR designs four filters to filter out ill-formatted credentials. Finally, HCR directly checks the validity of a set of non-sensitive credentials. We apply HCR to evaluate three representative types of NCCTs: Commercial NCCTs, open-source models, and chatbots with code completion capability. Our experimental results show that NCCTs can not only return the precise piece of their training data but also inadvertently leak additional secret strings. Notably, two valid credentials were identified during our experiments. Therefore, HCR raises a severe privacy concern about the potential leakage of hard-coded credentials in the training data of commercial NCCTs. All artifacts and data are released for future research purposes in https://github.com/HCR-Repo/HCR.

Your Code Secret Belongs to Me: Neural Code Completion Tools Can Memorize Hard-Coded Credentials

TL;DR

This work demonstrates that Neural Code Completion Tools (NCCTs) can memorize and occasionally reveal hard-coded credentials from their training data. It introduces the Hard-coded Credential Revealer (HCR), a phase-based framework that constructs prompts from GitHub-sourced secrets, applies regex-based identification, and uses entropy, pattern, and word filters to validate candidates, evaluating six NCCTs across 18 secret types. The results show NCCTs can reproduce exact training data and leak additional secrets, including two valid Stripe test keys, highlighting a substantial privacy risk. The authors discuss mitigations spanning data cleaning, privacy-preserving training, and post-processing, underscoring the need for auditing NCCTs before deployment and prompting further research into robust defenses.

Abstract

Neural Code Completion Tools (NCCTs) have reshaped the field of software engineering, which are built upon the language modeling technique and can accurately suggest contextually relevant code snippets. However, language models may emit the training data verbatim during inference with appropriate prompts. This memorization property raises privacy concerns of NCCTs about hard-coded credential leakage, leading to unauthorized access to applications, systems, or networks. Therefore, to answer whether NCCTs will emit the hard-coded credential, we propose an evaluation tool called Hard-coded Credential Revealer (HCR). HCR constructs test prompts based on GitHub code files with credentials to reveal the memorization phenomenon of NCCTs. Then, HCR designs four filters to filter out ill-formatted credentials. Finally, HCR directly checks the validity of a set of non-sensitive credentials. We apply HCR to evaluate three representative types of NCCTs: Commercial NCCTs, open-source models, and chatbots with code completion capability. Our experimental results show that NCCTs can not only return the precise piece of their training data but also inadvertently leak additional secret strings. Notably, two valid credentials were identified during our experiments. Therefore, HCR raises a severe privacy concern about the potential leakage of hard-coded credentials in the training data of commercial NCCTs. All artifacts and data are released for future research purposes in https://github.com/HCR-Repo/HCR.
Paper Structure (46 sections, 5 equations, 4 figures, 6 tables)

This paper contains 46 sections, 5 equations, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Memorization of Language Models
  4. Hard-Coded Credentials Leakage
  5. Threat Model & Ethics
  6. Threat Model
  7. Ethics Considerations
  8. HCR: Hard-coded Credential Revealer
  9. Phase 1: Identifying Secret Types
  10. Phase 2: Designing Regular Expressions for Secrets
  11. Phase 3: Constructing Prompts
  12. Phase 4: Querying NCCTs
  13. Phase 5: Validating Extracted Secrets
  14. Entropy Filter
  15. Pattern Filter
  16. ...and 31 more sections

Figures (4)

  • Figure 1: Illustration of the privacy attack to extract hard-coded credentials from NCCTs. The NCCTs have been trained on data with hard-coded credentials. Therefore, with prompts construsted by an adversary, they may emit the credential verbatim.
  • Figure 2: The overall framework of attacking with HCR in five phases.
  • Figure 3: We use different colors for "line_num", "token_num", "token_num_above", "token_num_below", "line_num_above", and "line_num_after". 'o' and 'x' markers refer to Copilot and CodeWhisperer correspondingly. Specifically, for hypothesis test 'mwu_$XY$', we test whether variable value in type $X$ is smaller than that in type $Y$.
  • Figure 4: The number of candidate secrets dropped by each filter applied in the order of the regex filter, the entropy filter, the pattern filter, and the word filter.