Analyzing Inference Privacy Risks Through Gradients in Machine Learning

TL;DR

The inefficacy of solely relying on data aggregation to achieve privacy against inference attacks in distributed learning is demonstrated, and a unified game-based framework that encompasses a broad range of attacks including attribute, property, distributional, and user disclosures is presented.

Abstract

In distributed learning settings, models are iteratively updated with shared gradients computed from potentially sensitive user data. While previous work has studied various privacy risks of sharing gradients, our paper aims to provide a systematic approach to analyze private information leakage from gradients. We present a unified game-based framework that encompasses a broad range of attacks including attribute, property, distributional, and user disclosures. We investigate how different uncertainties of the adversary affect their inferential power via extensive experiments on five datasets across various data modalities. Our results demonstrate the inefficacy of solely relying on data aggregation to achieve privacy against inference attacks in distributed learning. We further evaluate five types of defenses, namely, gradient pruning, signed gradient descent, adversarial perturbations, variational information bottleneck, and differential privacy, under both static and adaptive adversary settings. We provide an information-theoretic view for analyzing the effectiveness of these defenses against inference from gradients. Finally, we introduce a method for auditing attribute inference privacy, improving the empirical estimation of worst-case privacy through crafting adversarial canary records.

Figures (5)

• Figure 1: Overview of the unified inference game from gradients: the adversary infers the sensitive variable ${\bm{a}}$ from observations of the gradients $\Tilde{{\bm{g}}}$ computed on the private data batch ${\mathcal{D}}_{\bm{a}}$.
• Figure 2: Comparison of single-round and multi-round inference attacks on the Adult (AIA, PIA, DIA) and CREMA-D (UIA) datasets. A complete result on all datasets is provided in Appendix Figure \ref{['fig:sr_mr']}.
• Figure 3: Sensitivity analysis of the impact of varying batch sizes on the performance of inference attacks.
• Figure 4: Sensitivity analysis of the impact of varying model sizes on the performance of Property Inference Attack.
• Figure 5: Sensitivity analysis of the impact of adversary's knowledge on the performance of Property Inference Attack on the Adult dataset with a batch size of 16.

Theorems & Definitions (1)

• Definition 3.1