Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stop Tracking Me! Proactive Defense Against Attribute Inference Attack in LLMs

Dong Yan, Jian Liang, Ran He, Tieniu Tan

TL;DR

This work addresses privacy risks from attribute inference attacks on LLMs and the limitations of existing anonymization approaches. It introduces TRACE-RPS, a unified defense combining fine-grained anonymization (TRACE) with a light, two-stage optimization (RPS) that elicits model rejection to prevent inferences, with an alternative MPS strategy for stubborn instruction-following models. TRACE uses attention-guided privacy vocabulary extraction and inference-chain generation to pinpoint leakage, while RPS adds a minimal suffix to trigger refusals, yielding strong privacy protection with modest utility loss. Across open-source and closed-source models and multiple datasets, TRACE-RPS dramatically lowers attribute inference accuracy (often to below 5%), demonstrates cross-model generalization, and maintains favorable utility-privacy tradeoffs, highlighting a practical, user-controlled defense against privacy leakage in LLMs.

Abstract

Recent studies have shown that large language models (LLMs) can infer private user attributes (e.g., age, location, gender) from user-generated text shared online, enabling rapid and large-scale privacy breaches. Existing anonymization-based defenses are coarse-grained, lacking word-level precision in anonymizing privacy-leaking elements. Moreover, they are inherently limited as altering user text to hide sensitive cues still allows attribute inference to occur through models' reasoning capabilities. To address these limitations, we propose a unified defense framework that combines fine-grained anonymization (TRACE) with inference-preventing optimization (RPS). TRACE leverages attention mechanisms and inference chain generation to identify and anonymize privacy-leaking textual elements, while RPS employs a lightweight two-stage optimization strategy to induce model rejection behaviors, thereby preventing attribute inference. Evaluations across diverse LLMs show that TRACE-RPS reduces attribute inference accuracy from around 50\% to below 5\% on open-source models. In addition, our approach offers strong cross-model generalization, prompt-variation robustness, and utility-privacy tradeoffs. Our code is available at https://github.com/Jasper-Yan/TRACE-RPS.

Stop Tracking Me! Proactive Defense Against Attribute Inference Attack in LLMs

TL;DR

This work addresses privacy risks from attribute inference attacks on LLMs and the limitations of existing anonymization approaches. It introduces TRACE-RPS, a unified defense combining fine-grained anonymization (TRACE) with a light, two-stage optimization (RPS) that elicits model rejection to prevent inferences, with an alternative MPS strategy for stubborn instruction-following models. TRACE uses attention-guided privacy vocabulary extraction and inference-chain generation to pinpoint leakage, while RPS adds a minimal suffix to trigger refusals, yielding strong privacy protection with modest utility loss. Across open-source and closed-source models and multiple datasets, TRACE-RPS dramatically lowers attribute inference accuracy (often to below 5%), demonstrates cross-model generalization, and maintains favorable utility-privacy tradeoffs, highlighting a practical, user-controlled defense against privacy leakage in LLMs.

Abstract

Recent studies have shown that large language models (LLMs) can infer private user attributes (e.g., age, location, gender) from user-generated text shared online, enabling rapid and large-scale privacy breaches. Existing anonymization-based defenses are coarse-grained, lacking word-level precision in anonymizing privacy-leaking elements. Moreover, they are inherently limited as altering user text to hide sensitive cues still allows attribute inference to occur through models' reasoning capabilities. To address these limitations, we propose a unified defense framework that combines fine-grained anonymization (TRACE) with inference-preventing optimization (RPS). TRACE leverages attention mechanisms and inference chain generation to identify and anonymize privacy-leaking textual elements, while RPS employs a lightweight two-stage optimization strategy to induce model rejection behaviors, thereby preventing attribute inference. Evaluations across diverse LLMs show that TRACE-RPS reduces attribute inference accuracy from around 50\% to below 5\% on open-source models. In addition, our approach offers strong cross-model generalization, prompt-variation robustness, and utility-privacy tradeoffs. Our code is available at https://github.com/Jasper-Yan/TRACE-RPS.
Paper Structure (42 sections, 15 equations, 3 figures, 10 tables, 1 algorithm)

This paper contains 42 sections, 15 equations, 3 figures, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Attribute Inference Attack
  5. Attribute Inference Defense
  6. Methods
  7. Fine-Grained Anonymization
  8. Privacy Vocabulary Extraction.
  9. Privacy Inference Chain Generation and Guided Anonymization.
  10. Privacy-Preserving Optimization
  11. Rejection-Oriented Perturbation Search.
  12. Misattribute-Oriented Perturbation Search.
  13. Experiments
  14. Experimental Setup
  15. Main Results
  16. ...and 27 more sections

Figures (3)

  • Figure 1: Overview of the TRACE-RPS framework. Given user-generated texts, TRACE performs fine-grained anonymization through attention-based privacy vocabulary extraction and inference chain-guided editing to obscure sensitive attributes. RPS then applies privacy-preserving optimization to append a lightweight suffix that induces model rejection. Together, this unified defense framework proactively mitigates attribute inference attacks before the text is shared.
  • Figure 2: Robustness of RPS defense against 100 diverse inference prompts.
  • Figure 3: Transferability of RPS across models measured by attribute inference accuracy.