Thinking Outside of the Differential Privacy Box: A Case Study in Text Privatization with Language Model Prompting

TL;DR

This paper challenges the routine adoption of Differential Privacy in NLP by investigating DP-Prompt, a text-rewriting method that injects DP at generation, and comparing it to relaxed (Quasi-DP) and non-DP approaches. Through utility and privacy evaluations on the Blog Authorship Corpus, it shows that DP can improve semantic similarity and empirical privacy at strict budgets but often harms readability and overall practicality; results suggest rethinking DP's role in NLP and exploring alternative privacy-preserving strategies. The work highlights a nuanced DP-utility trade-off and calls for deeper theoretical and empirical work to identify when DP offers genuine value in text privatization. It underscores the need for hybrid or non-DP mechanisms that balance privacy guarantees with usable, natural language outputs.

Abstract

The field of privacy-preserving Natural Language Processing has risen in popularity, particularly at a time when concerns about privacy grow with the proliferation of Large Language Models. One solution consistently appearing in recent literature has been the integration of Differential Privacy (DP) into NLP techniques. In this paper, we take these approaches into critical view, discussing the restrictions that DP integration imposes, as well as bring to light the challenges that such restrictions entail. To accomplish this, we focus on $\textbf{DP-Prompt}$, a recent method for text privatization leveraging language models to rewrite texts. In particular, we explore this rewriting task in multiple scenarios, both with DP and without DP. To drive the discussion on the merits of DP in NLP, we conduct empirical utility and privacy experiments. Our results demonstrate the need for more discussion on the usability of DP in NLP and its benefits over non-DP approaches.