Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Just Rewrite It Again: A Post-Processing Method for Enhanced Semantic Similarity and Privacy Preservation of Differentially Private Rewritten Text

Stephen Meisenbacher, Florian Matthes

TL;DR

The paper tackles privacy risks in natural language processing by leveraging local differential privacy for text rewriting and introducing a post-processing step that rewrites DP-rewritten text again. It presents a mechanism-agnostic pipeline that uses Text2Text fine-tuning to realign DP outputs with their originals, with two tracks (basic and advanced) to balance utility and domain specificity. Empirical evaluations on Yelp and Trustpilot data show notable privacy gains against static and adaptive attackers, often accompanied by increased semantic similarity to the originals, albeit with some trade-offs and additional computational overhead. Overall, the approach extends the DP rewriting toolkit by exploiting the post-processing property to strengthen privacy protections while preserving, and in some cases enhancing, output usefulness for downstream tasks.

Abstract

The study of Differential Privacy (DP) in Natural Language Processing often views the task of text privatization as a $\textit{rewriting}$ task, in which sensitive input texts are rewritten to hide explicit or implicit private information. In order to evaluate the privacy-preserving capabilities of a DP text rewriting mechanism, $\textit{empirical privacy}$ tests are frequently employed. In these tests, an adversary is modeled, who aims to infer sensitive information (e.g., gender) about the author behind a (privatized) text. Looking to improve the empirical protections provided by DP rewriting methods, we propose a simple post-processing method based on the goal of aligning rewritten texts with their original counterparts, where DP rewritten texts are rewritten $\textit{again}$. Our results show that such an approach not only produces outputs that are more semantically reminiscent of the original inputs, but also texts which score on average better in empirical privacy evaluations. Therefore, our approach raises the bar for DP rewriting methods in their empirical privacy evaluations, providing an extra layer of protection against malicious adversaries.

Just Rewrite It Again: A Post-Processing Method for Enhanced Semantic Similarity and Privacy Preservation of Differentially Private Rewritten Text

TL;DR

The paper tackles privacy risks in natural language processing by leveraging local differential privacy for text rewriting and introducing a post-processing step that rewrites DP-rewritten text again. It presents a mechanism-agnostic pipeline that uses Text2Text fine-tuning to realign DP outputs with their originals, with two tracks (basic and advanced) to balance utility and domain specificity. Empirical evaluations on Yelp and Trustpilot data show notable privacy gains against static and adaptive attackers, often accompanied by increased semantic similarity to the originals, albeit with some trade-offs and additional computational overhead. Overall, the approach extends the DP rewriting toolkit by exploiting the post-processing property to strengthen privacy protections while preserving, and in some cases enhancing, output usefulness for downstream tasks.

Abstract

The study of Differential Privacy (DP) in Natural Language Processing often views the task of text privatization as a task, in which sensitive input texts are rewritten to hide explicit or implicit private information. In order to evaluate the privacy-preserving capabilities of a DP text rewriting mechanism, tests are frequently employed. In these tests, an adversary is modeled, who aims to infer sensitive information (e.g., gender) about the author behind a (privatized) text. Looking to improve the empirical protections provided by DP rewriting methods, we propose a simple post-processing method based on the goal of aligning rewritten texts with their original counterparts, where DP rewritten texts are rewritten . Our results show that such an approach not only produces outputs that are more semantically reminiscent of the original inputs, but also texts which score on average better in empirical privacy evaluations. Therefore, our approach raises the bar for DP rewriting methods in their empirical privacy evaluations, providing an extra layer of protection against malicious adversaries.
Paper Structure (36 sections, 2 equations, 2 figures, 1 table)

This paper contains 36 sections, 2 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Foundations
  3. Differentially Private Text Rewriting
  4. Local Differential Privacy
  5. Post-processing
  6. Empirical Privacy Evaluation
  7. Text2Text Generation
  8. A Post-processing Method for DP Rewritten Texts
  9. Preliminaries
  10. Method
  11. Corpus Creation
  12. LM Fine-Tuning
  13. Track Two: Further Fine-Tuning
  14. Data Rewriting and Release
  15. Experimental Setup
  16. ...and 21 more sections

Figures (2)

  • Figure 1: The Static and Adaptive Attackers. The adaptive attacker, with knowledge of the rewriting mechanism, generates "shadow" texts by rewriting publicly available texts from the same domain as the target "true" private texts.
  • Figure 2: A post-processing pipeline to improve the semantic coherence and privacy preservation of DP rewritten text. Both "tracks" involve a user creating a model T to rewrite DP rewritten text again. While a basic user utilizes large-scale public text corpora, the more advanced user (bottom track) also leverages domain-specific data to fine-tune T further to obtain T++.