Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multimodal Privacy-Preserving Entity Resolution with Fully Homomorphic Encryption

Susim Roy, Nalini Ratha

TL;DR

The paper tackles identity reconciliation in high-compliance domains under strict privacy constraints by proposing a multimodal framework that operates entirely in encrypted space using Fully Homomorphic Encryption (FHE). It combines biometric and biographic information via trainable BMMLP/BGMLP modules, trained with CLIP-based encoders, and performs encrypted enrollment and verification with the RNS-CKKS scheme, leveraging both feature-level and score-level fusion. The key contributions include a synthetic, scalable multimodal dataset, a complete encrypted pipeline for enrollment and verification, and empirical evidence that encrypted matching preserves accuracy while enabling privacy guarantees, along with notable latency benefits from parallelization. This approach offers a practical path toward secure, scalable identity resolution for institutions like DMVs and banks, enabling regulatory compliance without exposing sensitive Personal Identifiable Information during the matching lifecycle.

Abstract

The canonical challenge of entity resolution within high-compliance sectors, where secure identity reconciliation is frequently confounded by significant data heterogeneity, including syntactic variations in personal identifiers, is a longstanding and complex problem. To this end, we introduce a novel multimodal framework operating with the voluminous data sets typical of government and financial institutions. Specifically, our methodology is designed to address the tripartite challenge of data volume, matching fidelity, and privacy. Consequently, the underlying plaintext of personally identifiable information remains computationally inaccessible throughout the matching lifecycle, empowering institutions to rigorously satisfy stringent regulatory mandates with cryptographic assurances of client confidentiality while achieving a demonstrably low equal error rate and maintaining computational tractability at scale.

Multimodal Privacy-Preserving Entity Resolution with Fully Homomorphic Encryption

TL;DR

The paper tackles identity reconciliation in high-compliance domains under strict privacy constraints by proposing a multimodal framework that operates entirely in encrypted space using Fully Homomorphic Encryption (FHE). It combines biometric and biographic information via trainable BMMLP/BGMLP modules, trained with CLIP-based encoders, and performs encrypted enrollment and verification with the RNS-CKKS scheme, leveraging both feature-level and score-level fusion. The key contributions include a synthetic, scalable multimodal dataset, a complete encrypted pipeline for enrollment and verification, and empirical evidence that encrypted matching preserves accuracy while enabling privacy guarantees, along with notable latency benefits from parallelization. This approach offers a practical path toward secure, scalable identity resolution for institutions like DMVs and banks, enabling regulatory compliance without exposing sensitive Personal Identifiable Information during the matching lifecycle.

Abstract

The canonical challenge of entity resolution within high-compliance sectors, where secure identity reconciliation is frequently confounded by significant data heterogeneity, including syntactic variations in personal identifiers, is a longstanding and complex problem. To this end, we introduce a novel multimodal framework operating with the voluminous data sets typical of government and financial institutions. Specifically, our methodology is designed to address the tripartite challenge of data volume, matching fidelity, and privacy. Consequently, the underlying plaintext of personally identifiable information remains computationally inaccessible throughout the matching lifecycle, empowering institutions to rigorously satisfy stringent regulatory mandates with cryptographic assurances of client confidentiality while achieving a demonstrably low equal error rate and maintaining computational tractability at scale.
Paper Structure (9 sections, 1 equation, 3 figures, 3 tables)

This paper contains 9 sections, 1 equation, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Dataset and Pipeline Overview
  4. Training Procedure
  5. Encrypted Enrollment and Verification
  6. Experiments and Results
  7. Robustness of Verification and Identification
  8. Latency Analysis of FHE Operator
  9. Conclusion

Figures (3)

  • Figure 1: The above figure demonstrates the training pipeline to obtain the biometric and biographic score vectors. Here, $s^1_{bm}$ and $s^1_{bg}$ denote the score vectors for one unique individual.
  • Figure 2: The above figure demonstrates the encrypted enrollment and authentication pipeline. The three modality databases store the encrypted biometric, fused, and biographic templates, which are then matched during verification.
  • Figure 3: DET(a), ROC(b) and CMC(c) curves comparing verification and identification performance across modalities. The unimodal baselines (biometric and biographic) underperform relative to score-level and feature-level fusion. Figure (d) shows our method asymptotically approaching the theoretical maximum speedup possible with increasing threads. Note that the plaintext results are identical to the corresponding ciphertext results and hence are not shown.