Semantics-Preserved Distortion for Personal Privacy Protection in Information Management

TL;DR

This paper addresses privacy risks in NLP systems by proposing Semantics-Preserved Distortion, a data-centric approach that preserves semantic content while masking sensitive information. Central to the method is Neighboring Distribution Divergence (NDD), a metric that quantifies semantic disturbance caused by edits using MLM-based predictions, perplexity, and cosine similarity. The authors develop two distortion frameworks—Generative and Substitutive—and demonstrate through NER, constituency parsing, MRC, AIA defense, and a medical information management case that NDD-guided edits retain task performance better than naive masking and can strengthen defenses against privacy attacks. The work highlights practical privacy protection that complements existing structure-focused defenses and shows clear applicability to real-world medical data handling scenarios.

Abstract

In recent years, machine learning - particularly deep learning - has significantly impacted the field of information management. While several strategies have been proposed to restrict models from learning and memorizing sensitive information from raw texts, this paper suggests a more linguistically-grounded approach to distort texts while maintaining semantic integrity. To this end, we leverage Neighboring Distribution Divergence, a novel metric to assess the preservation of semantic meaning during distortion. Building on this metric, we present two distinct frameworks for semantic-preserving distortion: a generative approach and a substitutive approach. Our evaluations across various tasks, including named entity recognition, constituency parsing, and machine reading comprehension, affirm the plausibility and efficacy of our distortion technique in personal privacy protection. We also test our method against attribute attacks in three privacy-focused assignments within the NLP domain, and the findings underscore the simplicity and efficacy of our data-based improvement approach over structural improvement approaches. Moreover, we explore privacy protection in a specific medical information management scenario, showing our method effectively limits sensitive data memorization, underscoring its practicality.

Figures (1)

• Figure 1: Computation method for Neighboring Distribution Divergence.