Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LegalGuardian: A Privacy-Preserving Framework for Secure Integration of Large Language Models in Legal Practice

M. Mikail Demir, Hakan T. Otal, M. Abdullah Canbaz

TL;DR

The paper tackles confidentiality risks when deploying Large Language Models in legal practice by introducing LegalGuardian, a lightweight privacy-preserving framework that masks PII in prompts using Named Entity Recognition and local LLMs, enabling secure interaction with external AI tools. It combines synthetic data generation for immigration-law scenarios with a four-layer pipeline (data generation, PII masking, secure prompting, evaluation) and evaluates two masking strategies (GLiNER and Qwen2.5-14B) on 460 entities. Results show complementary strengths: GLiNER offers high precision and structured-entity accuracy, while Qwen2.5-14B provides higher recall and broader entity detection, with semantic fidelity largely preserved. The work demonstrates a practical path for lawyers to leverage AI while upholding client confidentiality, suggesting a hybrid approach and outlining future extensions to broader legal domains and privacy-preserving cloud integrations.

Abstract

Large Language Models (LLMs) hold promise for advancing legal practice by automating complex tasks and improving access to justice. However, their adoption is limited by concerns over client confidentiality, especially when lawyers include sensitive Personally Identifiable Information (PII) in prompts, risking unauthorized data exposure. To mitigate this, we introduce LegalGuardian, a lightweight, privacy-preserving framework tailored for lawyers using LLM-based tools. LegalGuardian employs Named Entity Recognition (NER) techniques and local LLMs to mask and unmask confidential PII within prompts, safeguarding sensitive data before any external interaction. We detail its development and assess its effectiveness using a synthetic prompt library in immigration law scenarios. Comparing traditional NER models with one-shot prompted local LLM, we find that LegalGuardian achieves a F1-score of 93% with GLiNER and 97% with Qwen2.5-14B in PII detection. Semantic similarity analysis confirms that the framework maintains high fidelity in outputs, ensuring robust utility of LLM-based tools. Our findings indicate that legal professionals can harness advanced AI technologies without compromising client confidentiality or the quality of legal documents.

LegalGuardian: A Privacy-Preserving Framework for Secure Integration of Large Language Models in Legal Practice

TL;DR

The paper tackles confidentiality risks when deploying Large Language Models in legal practice by introducing LegalGuardian, a lightweight privacy-preserving framework that masks PII in prompts using Named Entity Recognition and local LLMs, enabling secure interaction with external AI tools. It combines synthetic data generation for immigration-law scenarios with a four-layer pipeline (data generation, PII masking, secure prompting, evaluation) and evaluates two masking strategies (GLiNER and Qwen2.5-14B) on 460 entities. Results show complementary strengths: GLiNER offers high precision and structured-entity accuracy, while Qwen2.5-14B provides higher recall and broader entity detection, with semantic fidelity largely preserved. The work demonstrates a practical path for lawyers to leverage AI while upholding client confidentiality, suggesting a hybrid approach and outlining future extensions to broader legal domains and privacy-preserving cloud integrations.

Abstract

Large Language Models (LLMs) hold promise for advancing legal practice by automating complex tasks and improving access to justice. However, their adoption is limited by concerns over client confidentiality, especially when lawyers include sensitive Personally Identifiable Information (PII) in prompts, risking unauthorized data exposure. To mitigate this, we introduce LegalGuardian, a lightweight, privacy-preserving framework tailored for lawyers using LLM-based tools. LegalGuardian employs Named Entity Recognition (NER) techniques and local LLMs to mask and unmask confidential PII within prompts, safeguarding sensitive data before any external interaction. We detail its development and assess its effectiveness using a synthetic prompt library in immigration law scenarios. Comparing traditional NER models with one-shot prompted local LLM, we find that LegalGuardian achieves a F1-score of 93% with GLiNER and 97% with Qwen2.5-14B in PII detection. Semantic similarity analysis confirms that the framework maintains high fidelity in outputs, ensuring robust utility of LLM-based tools. Our findings indicate that legal professionals can harness advanced AI technologies without compromising client confidentiality or the quality of legal documents.
Paper Structure (18 sections, 5 figures, 4 tables)

This paper contains 18 sections, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Introduction to LegalGuardian Framework
  4. Methodology
  5. Overview of LegalGuardian Framework
  6. Synthetic Data and Content Generation (Green Box)
  7. PII Recognition & Masking Layer (Yellow Box)
  8. The Masking Process:
  9. Secure Prompting Layer (Purple Box)
  10. Secure Prompting Process:
  11. Evaluation Layer (Blue Box)
  12. Results & Findings
  13. Overall Accuracy
  14. Entity-Level Accuracy Metrics
  15. Semantic Similarity
  16. ...and 3 more sections

Figures (5)

  • Figure 1: Comparison of Legal Prompting Without (Left) and With (Right) the LegalGuardian Framework
  • Figure 2: Dataset Creation and Evaluation Pipeline for LegalGuardian Framework
  • Figure 3: Overall performance metrics for GLiNER and Qwen2.5-14B
  • Figure 4: Fake PII generation using LLM
  • Figure 5: Fake prompt generation for various tasks using LLM