Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploiting Unintended Feature Leakage in Collaborative Learning

Luca Melis, Congzheng Song, Emiliano De Cristofaro, Vitaly Shmatikov

TL;DR

The paper analyzes privacy risks in collaborative and federated learning where only model updates are exchanged. It introduces passive and active inference attacks that reveal membership and subset properties of participants’ data by exploiting gradient and embedding-layer information. Across two-party and multi-party settings, the attacks demonstrate strong leakage, including dynamic properties and person-specific inferences, with defenses proving largely ineffective in practical regimes. The findings underscore the need for new privacy-preserving techniques and detection mechanisms tailored to collaborative learning environments.

Abstract

Collaborative machine learning and related techniques such as federated learning allow multiple participants, each with his own training dataset, to build a joint model by training locally and periodically exchanging model updates. We demonstrate that these updates leak unintended information about participants' training data and develop passive and active inference attacks to exploit this leakage. First, we show that an adversarial participant can infer the presence of exact data points -- for example, specific locations -- in others' training data (i.e., membership inference). Then, we show how this adversary can infer properties that hold only for a subset of the training data and are independent of the properties that the joint model aims to capture. For example, he can infer when a specific person first appears in the photos used to train a binary gender classifier. We evaluate our attacks on a variety of tasks, datasets, and learning configurations, analyze their limitations, and discuss possible defenses.

Exploiting Unintended Feature Leakage in Collaborative Learning

TL;DR

The paper analyzes privacy risks in collaborative and federated learning where only model updates are exchanged. It introduces passive and active inference attacks that reveal membership and subset properties of participants’ data by exploiting gradient and embedding-layer information. Across two-party and multi-party settings, the attacks demonstrate strong leakage, including dynamic properties and person-specific inferences, with defenses proving largely ineffective in practical regimes. The findings underscore the need for new privacy-preserving techniques and detection mechanisms tailored to collaborative learning environments.

Abstract

Collaborative machine learning and related techniques such as federated learning allow multiple participants, each with his own training dataset, to build a joint model by training locally and periodically exchanging model updates. We demonstrate that these updates leak unintended information about participants' training data and develop passive and active inference attacks to exploit this leakage. First, we show that an adversarial participant can infer the presence of exact data points -- for example, specific locations -- in others' training data (i.e., membership inference). Then, we show how this adversary can infer properties that hold only for a subset of the training data and are independent of the properties that the joint model aims to capture. For example, he can infer when a specific person first appears in the photos used to train a binary gender classifier. We evaluate our attacks on a variety of tasks, datasets, and learning configurations, analyze their limitations, and discuss possible defenses.

Paper Structure

This paper contains 36 sections, 1 equation, 11 figures, 7 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Machine learning (ML)
  4. Collaborative learning
  5. Reasoning about Privacy in Machine Learning
  6. Inferring class representatives
  7. Inferring membership in training data
  8. Inferring properties of training data
  9. Inference Attacks
  10. Threat model
  11. Overview of the attacks
  12. Membership inference
  13. Passive property inference
  14. Active property inference
  15. Datasets and model architectures
  16. ...and 21 more sections

Figures (11)

  • Figure 1: Samples from a GAN attack on a gender classification model where the class is "female."
  • Figure 2: Overview of inference attacks against collaborative learning.
  • Figure 3: t-SNE projection of the features from different layers of the joint model on LFW gender classification; hollow circle point is female, solid triangle point is male, blue point is the property "race: black" and red point is data without the property.
  • Figure 4: AUC vs. the fraction of the batch that has the property on FaceScrub and Yelp-author.
  • Figure 5: Inferring occurrence of a single-batch property.
  • ...and 6 more figures