Membership Inference Attacks from Causal Principles

TL;DR

This work reframes Membership Inference Attack (MIA) evaluation as a causal problem, defining memorization as the causal effect of including a data point in training via $Y_i(1)-Y_i(0)$. It identifies and addresses biases intrinsic to one-run (training with interference) and zero-run (distribution shift confounding) regimes, deriving causal analogues of MIA metrics and practical estimators with non-asymptotic guarantees. The proposed estimators—employing IPW, G-Formula, and AIPW—provide identifiability under the respective regimes and align post-hoc evaluations with true memorization signals. Empirical results on synthetic data and CIFAR-10 demonstrate that correcting for interference and confounding yields reliable memorization measurements even when retraining is impractical. The framework thus offers a principled foundation for privacy evaluation in contemporary AI systems, with potential for extensions to subgroup analyses and other privacy attacks.

Abstract

Membership Inference Attacks (MIAs) are widely used to quantify training data memorization and assess privacy risks. Standard evaluation requires repeated retraining, which is computationally costly for large models. One-run methods (single training with randomized data inclusion) and zero-run methods (post hoc evaluation) are often used instead, though their statistical validity remains unclear. To address this gap, we frame MIA evaluation as a causal inference problem, defining memorization as the causal effect of including a data point in the training set. This novel formulation reveals and formalizes key sources of bias in existing protocols: one-run methods suffer from interference between jointly included points, while zero-run evaluations popular for LLMs are confounded by non-random membership assignment. We derive causal analogues of standard MIA metrics and propose practical estimators for multi-run, one-run, and zero-run regimes with non-asymptotic consistency guarantees. Experiments on real-world data show that our approach enables reliable memorization measurement even when retraining is impractical and under distribution shift, providing a principled foundation for privacy evaluation in modern AI systems.

Figures (4)

• Figure 1: Multi-run
• Figure 2: Causal graphs for the three different evaluation paradigms for MIAs. (a) In the multi-run setting, inclusion of a point is randomized, and for every point, a separate model is trained. (b) In the $1$-run setting, inclusion of every point is randomized, but only one model is trained. (c) In the $0$-run setting, data inclusion is not randomized, and the model is only trained once.
• Figure 3: ROC curves and corresponding AUCs for different MIA evaluation scenarios. \ref{['fig:roc_synthetic_ridge']} and \ref{['fig:roc_synthetic_dp_sgd']}: linear model trained on synthetic data, respectively with Ridge regression and DP-SGD. \ref{['fig:cifar_roc']}: ResNet model on CIFAR-10 data (averaged over 4 seeds).
• Figure 4: Additional metrics.(a) The loss distribution for drifted non-members is shifted, making them more distinguishable from members without true memorization. (b) The distribution of estimated ATE weights shows how our method corrects this by upweighting the minority of non-members that are distributionally similar to the training set.

Theorems & Definitions (20)

• Proposition 4.1
• Proposition 4.2
• Proposition 5.1
• Definition 5.2: Error stability; bousquet2002stability
• Definition 5.3: $\beta$-uniform training stability
• Theorem 5.4
• Remark 5.5: Extension to other MIAs
• Proposition 6.2
• Theorem 6.4
• Definition 1.3: Causal Comparison AUC