Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Thunder-DeID: Accurate and Efficient De-identification Framework for Korean Court Judgments

Sungeun Hahm, Heejin Kim, Gyuseong Lee, Hyunji Park, Jaejin Lee

TL;DR

Thunder-DeID tackles the challenge of de-identifying Korean court judgments at scale while complying with privacy laws. It introduces a DNN-based NER framework, a first Korean legal dataset with 6,700 judgments and 48,306 labeled entities, a three-tier PII taxonomy, and a Mecab-ko+BPE tokenizer, augmented by LLM-assisted data generation. The approach achieves state-of-the-art de-identification performance, with the Per-Epoch Entity Replacement strategy improving data diversity and robustness beyond baselines. This work enables scalable, legally compliant open access to judgments and provides a foundation for cross-jurisdictional privacy-preserving NLP in the legal domain.

Abstract

To ensure a balance between open access to justice and personal data protection, the South Korean judiciary mandates the de-identification of court judgments before they can be publicly disclosed. However, the current de-identification process is inadequate for handling court judgments at scale while adhering to strict legal requirements. Additionally, the legal definitions and categorizations of personal identifiers are vague and not well-suited for technical solutions. To tackle these challenges, we propose a de-identification framework called Thunder-DeID, which aligns with relevant laws and practices. Specifically, we (i) construct and release the first Korean legal dataset containing annotated judgments along with corresponding lists of entity mentions, (ii) introduce a systematic categorization of Personally Identifiable Information (PII), and (iii) develop an end-to-end deep neural network (DNN)-based de-identification pipeline. Our experimental results demonstrate that our model achieves state-of-the-art performance in the de-identification of court judgments.

Thunder-DeID: Accurate and Efficient De-identification Framework for Korean Court Judgments

TL;DR

Thunder-DeID tackles the challenge of de-identifying Korean court judgments at scale while complying with privacy laws. It introduces a DNN-based NER framework, a first Korean legal dataset with 6,700 judgments and 48,306 labeled entities, a three-tier PII taxonomy, and a Mecab-ko+BPE tokenizer, augmented by LLM-assisted data generation. The approach achieves state-of-the-art de-identification performance, with the Per-Epoch Entity Replacement strategy improving data diversity and robustness beyond baselines. This work enables scalable, legally compliant open access to judgments and provides a foundation for cross-jurisdictional privacy-preserving NLP in the legal domain.

Abstract

To ensure a balance between open access to justice and personal data protection, the South Korean judiciary mandates the de-identification of court judgments before they can be publicly disclosed. However, the current de-identification process is inadequate for handling court judgments at scale while adhering to strict legal requirements. Additionally, the legal definitions and categorizations of personal identifiers are vague and not well-suited for technical solutions. To tackle these challenges, we propose a de-identification framework called Thunder-DeID, which aligns with relevant laws and practices. Specifically, we (i) construct and release the first Korean legal dataset containing annotated judgments along with corresponding lists of entity mentions, (ii) introduce a systematic categorization of Personally Identifiable Information (PII), and (iii) develop an end-to-end deep neural network (DNN)-based de-identification pipeline. Our experimental results demonstrate that our model achieves state-of-the-art performance in the de-identification of court judgments.

Paper Structure

This paper contains 139 sections, 5 equations, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Medical de-identification.
  4. De-identification of court judgments.
  5. Methods
  6. Data Collection
  7. Annotation Scheme
  8. Placeholder Detection and Labeling
  9. PII Categories
  10. Replacement Lists
  11. Manual curation.
  12. Rule-based generation.
  13. Training Data Generation
  14. Tokenization
  15. Tokenization algorithm.
  16. ...and 124 more sections

Figures (4)

  • Figure 1: Overview of Thunder-DeID.
  • Figure 2: The three-tiered categorization scheme for PII in the domain of law and adjudication.
  • Figure 3: Tokenization and training data generation.
  • Figure B.1: Examples of court judgment data before and after annotation.