Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Re-identification of De-identified Documents with Autoregressive Infilling

Lucas Georges Gabriel Charpentier, Pierre Lison

TL;DR

The paper addresses the robustness of de-identification by constructing an automated re-identification adversary using a retrieval-augmented pipeline. A sparse retriever selects background documents, a dense retriever extracts passages relevant to each masked span, and an infilling model proposes re-identification hypotheses, iterating until all spans are replaced. Evaluations on Wikipedia biographies, TAB, and synthetic clinical notes show up to about 80% of masked spans can be recovered, with performance strongly increasing as background knowledge grows. Findings highlight the critical role of background knowledge and dataset characteristics, suggesting sanitizer designs should account for external information access and potential leakage; the framework provides a tool for stress-testing de-identification pipelines. Overall, the approach offers a structured way to probe privacy risks in sanitized texts and informs safer release practices.

Abstract

Documents revealing sensitive information about individuals must typically be de-identified. This de-identification is often done by masking all mentions of personally identifiable information (PII), thereby making it more difficult to uncover the identity of the person(s) in question. To investigate the robustness of de-identification methods, we present a novel, RAG-inspired approach that attempts the reverse process of re-identification based on a database of documents representing background knowledge. Given a text in which personal identifiers have been masked, the re-identification proceeds in two steps. A retriever first selects from the background knowledge passages deemed relevant for the re-identification. Those passages are then provided to an infilling model which seeks to infer the original content of each text span. This process is repeated until all masked spans are replaced. We evaluate the re-identification on three datasets (Wikipedia biographies, court rulings and clinical notes). Results show that (1) as many as 80% of de-identified text spans can be successfully recovered and (2) the re-identification accuracy increases along with the level of background knowledge.

Re-identification of De-identified Documents with Autoregressive Infilling

TL;DR

The paper addresses the robustness of de-identification by constructing an automated re-identification adversary using a retrieval-augmented pipeline. A sparse retriever selects background documents, a dense retriever extracts passages relevant to each masked span, and an infilling model proposes re-identification hypotheses, iterating until all spans are replaced. Evaluations on Wikipedia biographies, TAB, and synthetic clinical notes show up to about 80% of masked spans can be recovered, with performance strongly increasing as background knowledge grows. Findings highlight the critical role of background knowledge and dataset characteristics, suggesting sanitizer designs should account for external information access and potential leakage; the framework provides a tool for stress-testing de-identification pipelines. Overall, the approach offers a structured way to probe privacy risks in sanitized texts and informs safer release practices.

Abstract

Documents revealing sensitive information about individuals must typically be de-identified. This de-identification is often done by masking all mentions of personally identifiable information (PII), thereby making it more difficult to uncover the identity of the person(s) in question. To investigate the robustness of de-identification methods, we present a novel, RAG-inspired approach that attempts the reverse process of re-identification based on a database of documents representing background knowledge. Given a text in which personal identifiers have been masked, the re-identification proceeds in two steps. A retriever first selects from the background knowledge passages deemed relevant for the re-identification. Those passages are then provided to an infilling model which seeks to infer the original content of each text span. This process is repeated until all masked spans are replaced. We evaluate the re-identification on three datasets (Wikipedia biographies, court rulings and clinical notes). Results show that (1) as many as 80% of de-identified text spans can be successfully recovered and (2) the re-identification accuracy increases along with the level of background knowledge.
Paper Structure (61 sections, 1 figure, 15 tables)

This paper contains 61 sections, 1 figure, 15 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Text de-identification
  4. Text infilling
  5. Retrieval-augmented models
  6. Approach
  7. Sparse document retriever
  8. Dense passage retriever
  9. Infilling
  10. Evaluation
  11. Data
  12. Training details
  13. Final re-identification
  14. Metrics
  15. Sparse Retrieval
  16. ...and 46 more sections

Figures (1)

  • Figure 1: Sketch of the re-identification pipeline. The approach takes as input a document in which PII has been masked. A sparse retriever first selects relevant documents from the background knowledge. A dense retriever then extracts from those the chunks deemed most useful for re-identifying a particular text span. Finally, the infilling model produces a re-identification guess for that span given the retrieved chunks. The process is repeated until all text spans are filled back in.