How Private are Language Models in Abstractive Summarization?

TL;DR

This paper investigates privacy risks in abstractive summarization by language models in sensitive medical and legal domains. It introduces pseudonymized datasets with expert-annotated gold-standard anonymous summaries and compares four open-weight and two closed models across prompting strategies and instruction fine-tuning (IFT). The findings show that LMs frequently leak personal information in summaries, while expert human summaries provide substantially higher privacy protection; IFT on pseudonymized data markedly improves privacy preservation and can narrow the gap with frontier models, especially for medical tasks. The work highlights practical implications for privacy-preserving data sharing and suggests that anonymize-after-summarize is a robust strategy, with privacy performance varying by PII type and model, guiding safer deployment of LM-based summarization in sensitive domains.

Abstract

In sensitive domains such as medical and legal, protecting sensitive information is critical, with protective laws strictly prohibiting the disclosure of personal data. This poses challenges for sharing valuable data such as medical reports and legal cases summaries. While language models (LMs) have shown strong performance in text summarization, it is still an open question to what extent they can provide privacy-preserving summaries from non-private source documents. In this paper, we perform a comprehensive study of privacy risks in LM-based summarization across two closed- and four open-weight models of different sizes and families. We experiment with both prompting and fine-tuning strategies for privacy-preservation across a range of summarization datasets including medical and legal domains. Our quantitative and qualitative analysis, including human evaluation, shows that LMs frequently leak personally identifiable information in their summaries, in contrast to human-generated privacy-preserving summaries, which demonstrate significantly higher privacy protection levels. These findings highlight a substantial gap between current LM capabilities and expert human expert performance in privacy-sensitive summarization tasks.

Figures (8)

• Figure 1: Prompting GPT-4o to generate a private summary of a clinical text. Orange represents leaked PII.
• Figure 2: An overview of the pseudonymization process.
• Figure 3: Prompt templates for summarization.
• Figure 4: Results of the private summary experiments. Top two rows display summarization quality metrics, while bottom three rows present privacy metrics. All metrics are averaged across prompt variations and PII types.
• Figure 5: Prompt for PII detection