Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Membership Inference for Contrastive Pre-training Models with Text-only PII Queries

Ruoxi Cheng, Yizhong Ding, Hongyi Zhang, Yiyan Huang

Abstract

Contrastive pretraining models such as CLIP and CLAP underpin many vision-language and audio-language systems, yet their reliance on web-scale data raises growing concerns about memorizing Personally Identifiable Information (PII). Auditing such models via membership inference is challenging in practice: shadow-model MIAs are computationally prohibitive for large multimodal backbones, and existing multimodal attacks typically require querying the target with paired biometric inputs, thereby directly exposing sensitive biometric information to the target model. We propose Unimodal Membership Inference Detector (UMID), a text-only auditing framework that performs text-guided cross-modal latent inversion and extracts two complementary signals, similarity (alignment to the queried text) and variability (consistency across randomized inversions). UMID compares these statistics to a lightweight non-member reference constructed from synthetic gibberish and makes decisions via an ensemble of unsupervised anomaly detectors. Comprehensive experiments across diverse CLIP and CLAP architectures demonstrate that UMID significantly improves the effectiveness and efficiency over prior MIAs, delivering strong detection performance with sub-second auditing cost while complying with realistic privacy constraints.

Membership Inference for Contrastive Pre-training Models with Text-only PII Queries

Abstract

Contrastive pretraining models such as CLIP and CLAP underpin many vision-language and audio-language systems, yet their reliance on web-scale data raises growing concerns about memorizing Personally Identifiable Information (PII). Auditing such models via membership inference is challenging in practice: shadow-model MIAs are computationally prohibitive for large multimodal backbones, and existing multimodal attacks typically require querying the target with paired biometric inputs, thereby directly exposing sensitive biometric information to the target model. We propose Unimodal Membership Inference Detector (UMID), a text-only auditing framework that performs text-guided cross-modal latent inversion and extracts two complementary signals, similarity (alignment to the queried text) and variability (consistency across randomized inversions). UMID compares these statistics to a lightweight non-member reference constructed from synthetic gibberish and makes decisions via an ensemble of unsupervised anomaly detectors. Comprehensive experiments across diverse CLIP and CLAP architectures demonstrate that UMID significantly improves the effectiveness and efficiency over prior MIAs, delivering strong detection performance with sub-second auditing cost while complying with realistic privacy constraints.
Paper Structure (45 sections, 3 theorems, 6 equations, 5 figures, 6 tables, 1 algorithm)

This paper contains 45 sections, 3 theorems, 6 equations, 5 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related Work
  4. Contrastive pretraining models
  5. PII leakage in MLLMs
  6. Membership inference attacks
  7. Method
  8. Problem Formulation
  9. Auditing scenario
  10. Detector's capability
  11. UMID: Unimodal Membership Inference Detector
  12. Feature Extraction via Latent Inversion
  13. Geometric Separation with Theoretical Justification
  14. Membership Inference via Anomaly Detection
  15. Semantic-null baseline
  16. ...and 30 more sections

Key Result

Theorem 3.1

Given a member text $t_{\mathrm{in}}$ and a non-member text $t_{\mathrm{out}}$ satisfying the geometric properties above. Let $\Delta_S:=S_\infty(t_{\mathrm{in}})-S_\infty(t_{\mathrm{out}})$ and $\Delta_D:=D_\infty^2(t_{\mathrm{out}})-D_\infty^2(t_{\mathrm{in}})$ be the population gaps, and define $

Figures (5)

  • Figure 1: Comparison of UMID and traditional MIA methods. Traditional approaches rely on shadow models or bimodal input, while UMID turns membership inference problem into anomaly detection using text-only queries.
  • Figure 2: Visualization of geometric separation. The extracted similarity and variability features via latent inversion exhibit a clear distributional gap between samples within and outside the training dataset of the target model.
  • Figure 3: Pipeline of UMID. We employ an optimizer guided by target model to align non-text embeddings with PII text embeddings, maximizing their cosine similarity. By analyzing similarity and variability features of these optimized samples relative to the synthetic gibberish baseline, an anomaly detection system identifies abnormal patterns to infer the membership of the input text.
  • Figure 4: Detection accuracy for CLIP model (ResNet-50) under various parameters.
  • Figure 5: Detection accuracy for CLAP model (LibriSpeech) under various parameters.

Theorems & Definitions (5)

  • Theorem 3.1: Finite-sample geometric separation
  • Proposition B.4: Population separation gaps
  • proof
  • Lemma B.5: Concentration Bounds
  • proof