Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PrivacyScalpel: Enhancing LLM Privacy via Interpretable Feature Intervention with Sparse Autoencoders

Ahmed Frikha, Muhammad Reza Ar Razi, Krishna Kanth Nakka, Ricardo Mendes, Xue Jiang, Xuebing Zhou

TL;DR

PrivacyScalpel addresses PII memorization in LLMs by coupling interpretable feature discovery with targeted, low-utility-interference interventions. It identifies PII-rich representations via layer probing, disentangles them with a k-Sparse Autoencoder, and applies either feature ablation or vector steering to suppress leakage while preserving task performance. Empirical results on Gemma2-2b and Llama2-7b show email leakage drops from 5.15% to 0% while maintaining over 99.4% of original utility, outperforming neuron-focused defenses. The approach also yields mechanistic insights into PII memorization, advancing interpretable, privacy-preserving AI deployment for real-world applications.

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language processing but also pose significant privacy risks by memorizing and leaking Personally Identifiable Information (PII). Existing mitigation strategies, such as differential privacy and neuron-level interventions, often degrade model utility or fail to effectively prevent leakage. To address this challenge, we introduce PrivacyScalpel, a novel privacy-preserving framework that leverages LLM interpretability techniques to identify and mitigate PII leakage while maintaining performance. PrivacyScalpel comprises three key steps: (1) Feature Probing, which identifies layers in the model that encode PII-rich representations, (2) Sparse Autoencoding, where a k-Sparse Autoencoder (k-SAE) disentangles and isolates privacy-sensitive features, and (3) Feature-Level Interventions, which employ targeted ablation and vector steering to suppress PII leakage. Our empirical evaluation on Gemma2-2b and Llama2-7b, fine-tuned on the Enron dataset, shows that PrivacyScalpel significantly reduces email leakage from 5.15\% to as low as 0.0\%, while maintaining over 99.4\% of the original model's utility. Notably, our method outperforms neuron-level interventions in privacy-utility trade-offs, demonstrating that acting on sparse, monosemantic features is more effective than manipulating polysemantic neurons. Beyond improving LLM privacy, our approach offers insights into the mechanisms underlying PII memorization, contributing to the broader field of model interpretability and secure AI deployment.

PrivacyScalpel: Enhancing LLM Privacy via Interpretable Feature Intervention with Sparse Autoencoders

TL;DR

PrivacyScalpel addresses PII memorization in LLMs by coupling interpretable feature discovery with targeted, low-utility-interference interventions. It identifies PII-rich representations via layer probing, disentangles them with a k-Sparse Autoencoder, and applies either feature ablation or vector steering to suppress leakage while preserving task performance. Empirical results on Gemma2-2b and Llama2-7b show email leakage drops from 5.15% to 0% while maintaining over 99.4% of original utility, outperforming neuron-focused defenses. The approach also yields mechanistic insights into PII memorization, advancing interpretable, privacy-preserving AI deployment for real-world applications.

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language processing but also pose significant privacy risks by memorizing and leaking Personally Identifiable Information (PII). Existing mitigation strategies, such as differential privacy and neuron-level interventions, often degrade model utility or fail to effectively prevent leakage. To address this challenge, we introduce PrivacyScalpel, a novel privacy-preserving framework that leverages LLM interpretability techniques to identify and mitigate PII leakage while maintaining performance. PrivacyScalpel comprises three key steps: (1) Feature Probing, which identifies layers in the model that encode PII-rich representations, (2) Sparse Autoencoding, where a k-Sparse Autoencoder (k-SAE) disentangles and isolates privacy-sensitive features, and (3) Feature-Level Interventions, which employ targeted ablation and vector steering to suppress PII leakage. Our empirical evaluation on Gemma2-2b and Llama2-7b, fine-tuned on the Enron dataset, shows that PrivacyScalpel significantly reduces email leakage from 5.15\% to as low as 0.0\%, while maintaining over 99.4\% of the original model's utility. Notably, our method outperforms neuron-level interventions in privacy-utility trade-offs, demonstrating that acting on sparse, monosemantic features is more effective than manipulating polysemantic neurons. Beyond improving LLM privacy, our approach offers insights into the mechanisms underlying PII memorization, contributing to the broader field of model interpretability and secure AI deployment.

Paper Structure

This paper contains 29 sections, 8 equations, 1 figure, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Privacy Risks in LLMs
  4. Privacy-Preserving Methods in LLMs
  5. Interpretability in LLMs Using Sparse Autoencoders
  6. Methodology
  7. Probing for PII Features
  8. Training the k-Sparse Autoencoder
  9. Defense Method
  10. Feature Ablation
  11. Feature Vector Steering
  12. Probing for Latent Features
  13. Top-k Probing for Latent Features
  14. Difference-in-Means Vectors
  15. Experiments
  16. ...and 14 more sections

Figures (1)

  • Figure 1: Overview of our framework. Given an input prompt, we extract the activations of input tokens at layer $l$ and perform intervention. We pass the activations through the SAE encoder and intervene on the SAE encoded high-dimensional features through Feature Ablation and Steering. We then decode the intervened features back to the original embedding space.