Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive PII Mitigation Framework for Large Language Models

Shubhi Asthana, Ruchi Mahindru, Bing Zhang, Jorge Sanz

TL;DR

The paper tackles the challenge of safely deploying LLMs under diverse data privacy laws (GDPR, CCPA, PIPEDA) by proposing an adaptive PII mitigation framework. The framework combines an Adaptive Policy Engine, Dynamic Contextual PII Detection, and Adaptive Masking with real-time policy alignment to GRC systems. Key results show a Passport Number F1 of 0.95, outperforming Microsoft Presidio (0.33) and Amazon Comprehend (0.54), and a human trust rating of 4.6/5, with GDPR prompting stricter anonymization than CCPA. This work provides a scalable solution for enterprise privacy compliance in LLM pipelines, including training data, outputs, and prompt data, integrated via OneShield Guardrails.

Abstract

Artificial Intelligence (AI) faces growing challenges from evolving data protection laws and enforcement practices worldwide. Regulations like GDPR and CCPA impose strict compliance requirements on Machine Learning (ML) models, especially concerning personal data use. These laws grant individuals rights such as data correction and deletion, complicating the training and deployment of Large Language Models (LLMs) that rely on extensive datasets. Public data availability does not guarantee its lawful use for ML, amplifying these challenges. This paper introduces an adaptive system for mitigating risk of Personally Identifiable Information (PII) and Sensitive Personal Information (SPI) in LLMs. It dynamically aligns with diverse regulatory frameworks and integrates seamlessly into Governance, Risk, and Compliance (GRC) systems. The system uses advanced NLP techniques, context-aware analysis, and policy-driven masking to ensure regulatory compliance. Benchmarks highlight the system's effectiveness, with an F1 score of 0.95 for Passport Numbers, outperforming tools like Microsoft Presidio (0.33) and Amazon Comprehend (0.54). In human evaluations, the system achieved an average user trust score of 4.6/5, with participants acknowledging its accuracy and transparency. Observations demonstrate stricter anonymization under GDPR compared to CCPA, which permits pseudonymization and user opt-outs. These results validate the system as a scalable and robust solution for enterprise privacy compliance.

Adaptive PII Mitigation Framework for Large Language Models

TL;DR

The paper tackles the challenge of safely deploying LLMs under diverse data privacy laws (GDPR, CCPA, PIPEDA) by proposing an adaptive PII mitigation framework. The framework combines an Adaptive Policy Engine, Dynamic Contextual PII Detection, and Adaptive Masking with real-time policy alignment to GRC systems. Key results show a Passport Number F1 of 0.95, outperforming Microsoft Presidio (0.33) and Amazon Comprehend (0.54), and a human trust rating of 4.6/5, with GDPR prompting stricter anonymization than CCPA. This work provides a scalable solution for enterprise privacy compliance in LLM pipelines, including training data, outputs, and prompt data, integrated via OneShield Guardrails.

Abstract

Artificial Intelligence (AI) faces growing challenges from evolving data protection laws and enforcement practices worldwide. Regulations like GDPR and CCPA impose strict compliance requirements on Machine Learning (ML) models, especially concerning personal data use. These laws grant individuals rights such as data correction and deletion, complicating the training and deployment of Large Language Models (LLMs) that rely on extensive datasets. Public data availability does not guarantee its lawful use for ML, amplifying these challenges. This paper introduces an adaptive system for mitigating risk of Personally Identifiable Information (PII) and Sensitive Personal Information (SPI) in LLMs. It dynamically aligns with diverse regulatory frameworks and integrates seamlessly into Governance, Risk, and Compliance (GRC) systems. The system uses advanced NLP techniques, context-aware analysis, and policy-driven masking to ensure regulatory compliance. Benchmarks highlight the system's effectiveness, with an F1 score of 0.95 for Passport Numbers, outperforming tools like Microsoft Presidio (0.33) and Amazon Comprehend (0.54). In human evaluations, the system achieved an average user trust score of 4.6/5, with participants acknowledging its accuracy and transparency. Observations demonstrate stricter anonymization under GDPR compared to CCPA, which permits pseudonymization and user opt-outs. These results validate the system as a scalable and robust solution for enterprise privacy compliance.
Paper Structure (24 sections, 4 figures, 4 tables)

This paper contains 24 sections, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Motivation for an Adaptive Framework
  3. Background and Motivation
  4. Legal and Ethical Implications
  5. Technical Challenges
  6. Proposed Method
  7. Adaptive Policy Engine
  8. Dynamic Contextual PII Detection
  9. Contextual Sensitivity Scoring
  10. Adaptive Masking and Anonymization Techniques
  11. Application and Use Cases
  12. LLM Training Data
  13. LLM Outputs
  14. Prompt data in LLM interactions
  15. Integration of OneShield Guardrails
  16. ...and 9 more sections

Figures (4)

  • Figure 1: Adaptive PII mitigation framework
  • Figure 2: Levels of PII sensitivity
  • Figure 3: OneShield Guardrails framework
  • Figure 4: OneShield Guardrails deployment - Adaptive PII system framework