Sequential Membership Inference Attacks

TL;DR

An `optimal'MI attack, SeMI*, is developed that uses the sequence of model updates to identify the presence of a target inserted at a certain update step, while accessing a finite number of samples with or without privacy.

Abstract

Modern AI models are not static. They go through multiple updates in their lifecycles. Thus, exploiting the model dynamics to create stronger Membership Inference (MI) attacks and tighter privacy audits are timely questions. Though the literature empirically shows that using a sequence of model updates can increase the power of MI attacks, rigorous analysis of the `optimal' MI attacks is limited to static models with infinite samples. Hence, we develop an `optimal' MI attack, SeMI*, that uses the sequence of model updates to identify the presence of a target inserted at a certain update step. For the empirical mean computation, we derive the optimal power of SeMI*, while accessing a finite number of samples with or without privacy. Our results retrieve the existing asymptotic analysis. We observe that having access to the model sequence avoids the dilution of MI signals unlike the existing attacks on the final model, where the MI signal vanishes as training data accumulates. Furthermore, an adversary can use SeMI* to tune both the insertion time and the canary to yield tighter privacy audits. Finally, we conduct experiments across data distributions and models trained or fine-tuned with DP-SGD demonstrating that practical variants of SeMI* lead to tighter privacy audits than the baselines.

Figures (10)

• Figure 1: Log-likelihood ratio statistics as a function of observed updates ($n=10$, $\tau=5$). (a) $\mathrm{SeMI}^*$: separation appears at $\tau$ and remains constant. (b) Final Observation: separation appears at $\tau$ but decreases as $T$ grows.
• Figure 2: Comparison of sequential tests ($n=10$, $z^*$ with Mahalanobis distance 3). (a) Power (TPR at 1% FPR) vs number of updates $T$. (b) Power vs target Mahalanobis distance ($T=5$).
• Figure 3: Privacy auditing of DP-SGD fine-tuning. Estimated $\varepsilon$ (lower bound) versus ground truth $\varepsilon$ for different attacks, averaged over insertion times $\tau \in \{1, \ldots, T\}$. Closer to the diagonal (ideal) is better. $\mathrm{SeMI}^{\mathrm{SGD}}$ yields tighter lower bounds than heuristic baselines.
• Figure 4: ROC curves comparing $\mathrm{SeMI}^*$, $\mathrm{SeMI}^{\mathrm{Unif}}$, $\mathrm{SeMI}^{\max}$, and Final Observation for different $T$.
• Figure 5: Composite hypothesis setting with $\tau \sim \mathrm{Uniform}(1, T)$. (a) ROC curves comparing $\mathrm{SeMI}^{\max}$, $\mathrm{SeMI}^*$, and Final Observation. (b) Power vs Mahalanobis distance.

Theorems & Definitions (69)

• Lemma 2.1: Connecting MI Test Error and Privacy Audits kairouz2015composition
• Theorem 3.1: Multivariate Log Likelihood Ratio (LR)
• Lemma 3.1
• Remark 3.2: Isolation Property
• Theorem 3.3: Log Likelihood Ratio
• Lemma 3.3: Type I Error
• Lemma 3.3: Type II Error
• Lemma 3.3: Final Observation Type I Error
• Lemma 3.3: Final Observation Type II Error
• Remark 3.4: Adapting to Distribution Shifts