Imitative Membership Inference Attack

TL;DR

This work addresses the computational burden of membership inference attacks by introducing IMIA, a target-informed approach that trains a limited set of imitative out/in models to mirror a target model's behavior. By using a two-stage imitative training regime and a non-parametric inference strategy, IMIA achieves superior attack performance across diverse datasets and both non-adaptive and adaptive threat models while using far fewer shadow models. The paper provides extensive ablations and demonstrates robustness to defenses like DP-SGD and to distribution shifts, highlighting practical privacy implications. Overall, IMIA offers a scalable and effective framework for privacy auditing in neural networks and sets a new benchmark for MIAs under constrained computational budgets.

Abstract

A Membership Inference Attack (MIA) assesses how much a target machine learning model reveals about its training data by determining whether specific query instances were part of the training set. State-of-the-art MIAs rely on training hundreds of shadow models that are independent of the target model, leading to significant computational overhead. In this paper, we introduce Imitative Membership Inference Attack (IMIA), which employs a novel imitative training technique to strategically construct a small number of target-informed imitative models that closely replicate the target model's behavior for inference. Extensive experimental results demonstrate that IMIA substantially outperforms existing MIAs in various attack settings while only requiring less than 5% of the computational cost of state-of-the-art approaches.

Figures (19)

• Figure 1: Distributions of membership scores (i.e., scaled confidence scores, defined in \ref{['sec:imia_framework']}) for six CIFAR-100 instances with varying attack difficulty (easy, medium, hard). A larger overlap between in (trained with the instance) and out (trained without the instance) score distributions indicates greater difficulty in determining membership. The dashed vertical line represents each instance’s score on the target model. Top row: target-agnostic shadow models show high predictive variance (with long tails and wide distributions) for both members and non-members, resulting in significant overlap that hampers reliable inference, especially for hard-to-attack instances. Bottom row: target-informed imitative models exhibit more stable and well-separated distributions, enabling effective inference across all levels of difficulty. More examples are in \ref{['fig:demo_model_stability_appendix']}. Best viewed in color.
• Figure 2: Demonstration of $\mathsf{IMIA}$ in the non-adaptive setting. The adversary ① constructs imitative in and out models by mimicking the behaviors of the target model $f_\theta$, ② performs membership inference on query instance $(x,y)$ using trained imitative models and selected proxy data.
• Figure 3: Performance on MNIST under varying computational budgets. $\mathsf{IMIA}$$_\text{Gaussian}$ employs a Gaussian likelihood ratio sp22lira on imitative models to compute the final scores.
• Figure 4: Normalized residual distributions (defined in \ref{['sec:exp_imiative']}) of shadow vs. imitative models on CIFAR-100. The residuals of imitative models closely follow the standard normal distribution, indicating they better mimic the target model’s behavior.
• Figure 5: The impact of architecture differences between the target model and the imitative models trained on CIFAR-100.

Theorems & Definitions (1)

• Definition 1: Canonical Membership Inference Security Game