Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PIG: Privacy Jailbreak Attack on LLMs via Gradient-based Iterative In-Context Optimization

Yidan Wang, Yanan Cao, Yubing Ren, Fang Fang, Zheng Lin, Binxing Fang

TL;DR

This paper proposes PIG, a novel framework targeting Personally Identifiable Information (PII) and addressing the limitations of current jailbreak methods, and shows that PIG outperforms baseline methods and achieves state-of-the-art (SoTA) results.

Abstract

Large Language Models (LLMs) excel in various domains but pose inherent privacy risks. Existing methods to evaluate privacy leakage in LLMs often use memorized prefixes or simple instructions to extract data, both of which well-alignment models can easily block. Meanwhile, Jailbreak attacks bypass LLM safety mechanisms to generate harmful content, but their role in privacy scenarios remains underexplored. In this paper, we examine the effectiveness of jailbreak attacks in extracting sensitive information, bridging privacy leakage and jailbreak attacks in LLMs. Moreover, we propose PIG, a novel framework targeting Personally Identifiable Information (PII) and addressing the limitations of current jailbreak methods. Specifically, PIG identifies PII entities and their types in privacy queries, uses in-context learning to build a privacy context, and iteratively updates it with three gradient-based strategies to elicit target PII. We evaluate PIG and existing jailbreak methods using two privacy-related datasets. Experiments on four white-box and two black-box LLMs show that PIG outperforms baseline methods and achieves state-of-the-art (SoTA) results. The results underscore significant privacy risks in LLMs, emphasizing the need for stronger safeguards. Our code is availble at https://github.com/redwyd/PrivacyJailbreak.

PIG: Privacy Jailbreak Attack on LLMs via Gradient-based Iterative In-Context Optimization

TL;DR

This paper proposes PIG, a novel framework targeting Personally Identifiable Information (PII) and addressing the limitations of current jailbreak methods, and shows that PIG outperforms baseline methods and achieves state-of-the-art (SoTA) results.

Abstract

Large Language Models (LLMs) excel in various domains but pose inherent privacy risks. Existing methods to evaluate privacy leakage in LLMs often use memorized prefixes or simple instructions to extract data, both of which well-alignment models can easily block. Meanwhile, Jailbreak attacks bypass LLM safety mechanisms to generate harmful content, but their role in privacy scenarios remains underexplored. In this paper, we examine the effectiveness of jailbreak attacks in extracting sensitive information, bridging privacy leakage and jailbreak attacks in LLMs. Moreover, we propose PIG, a novel framework targeting Personally Identifiable Information (PII) and addressing the limitations of current jailbreak methods. Specifically, PIG identifies PII entities and their types in privacy queries, uses in-context learning to build a privacy context, and iteratively updates it with three gradient-based strategies to elicit target PII. We evaluate PIG and existing jailbreak methods using two privacy-related datasets. Experiments on four white-box and two black-box LLMs show that PIG outperforms baseline methods and achieves state-of-the-art (SoTA) results. The results underscore significant privacy risks in LLMs, emphasizing the need for stronger safeguards. Our code is availble at https://github.com/redwyd/PrivacyJailbreak.
Paper Structure (49 sections, 4 equations, 6 figures, 4 tables, 1 algorithm)

This paper contains 49 sections, 4 equations, 6 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Privacy Leakage
  4. Jailbreak Attack
  5. Preliminaries
  6. Large Language Models (LLM)
  7. LLM Privacy Jailbreak Attack
  8. Method
  9. Threat Model
  10. PII Identification
  11. Privacy In-Context Learning
  12. Gradient-based Iterative Optimization
  13. Random Strategy
  14. Entity Strategy
  15. Dynamic Strategy
  16. ...and 34 more sections

Figures (6)

  • Figure 1: An example of privacy jailbreak attack via ICL, different contexts yield varying results.
  • Figure 2: The overview of our proposed framework PIG for privacy jailbreak attack.
  • Figure 3: The ASR of ICL under different few-shots.
  • Figure 4: The top compares initialization loss, and the bottom shows ASR for GCG and PIG over 100 epochs.
  • Figure 5: Comparison of ASR and loss curves for GCG and PIG over epochs, with PIG using three strategies.
  • ...and 1 more figures