Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SoK: Challenges in Tabular Membership Inference Attacks

Cristina Pêra, Tânia Carvalho, Maxime Cordy, Luís Antunes

TL;DR

This paper investigates Membership Inference Attacks (MIAs) in tabular data across centralized and Federated Learning (FL) settings, identifying gaps in prior work and proposing a taxonomy based on attacker access. It conducts extensive empirical evaluations using shadow-model and surrogate (LiRA/RMIA) attacks, plus metric-based approaches, across nine datasets, and examines defenses including regularization, differential privacy, and secure aggregation. The findings show generally poor MIA performance on tabular data, but reveal pronounced vulnerability for single-outs and strong transferability of attacks across surrogate models, with outsider FL attacks posing nontrivial privacy risks. Overall, the work highlights the need for robust, dataset-specific privacy auditing and defense strategies in tabular settings and FL, and points to future directions such as data augmentation and adaptive attack strategies to improve MIA assessments.

Abstract

Membership Inference Attacks (MIAs) are currently a dominant approach for evaluating privacy in machine learning applications. Despite their significance in identifying records belonging to the training dataset, several concerns remain unexplored, particularly with regard to tabular data. In this paper, first, we provide an extensive review and analysis of MIAs considering two main learning paradigms: centralized and federated learning. We extend and refine the taxonomy for both. Second, we demonstrate the efficacy of MIAs in tabular data using several attack strategies, also including defenses. Furthermore, in a federated learning scenario, we consider the threat posed by an outsider adversary, which is often neglected. Third, we demonstrate the high vulnerability of single-outs (records with a unique signature) to MIAs. Lastly, we explore how MIAs transfer across model architectures. Our results point towards a general poor performance of these attacks in tabular data which contrasts with previous state-of-the-art. Notably, even attacks with limited attack performance can still successfully expose a large portion of single-outs. Moreover, our findings suggest that using different surrogate models makes MIAs more effective.

SoK: Challenges in Tabular Membership Inference Attacks

TL;DR

This paper investigates Membership Inference Attacks (MIAs) in tabular data across centralized and Federated Learning (FL) settings, identifying gaps in prior work and proposing a taxonomy based on attacker access. It conducts extensive empirical evaluations using shadow-model and surrogate (LiRA/RMIA) attacks, plus metric-based approaches, across nine datasets, and examines defenses including regularization, differential privacy, and secure aggregation. The findings show generally poor MIA performance on tabular data, but reveal pronounced vulnerability for single-outs and strong transferability of attacks across surrogate models, with outsider FL attacks posing nontrivial privacy risks. Overall, the work highlights the need for robust, dataset-specific privacy auditing and defense strategies in tabular settings and FL, and points to future directions such as data augmentation and adaptive attack strategies to improve MIA assessments.

Abstract

Membership Inference Attacks (MIAs) are currently a dominant approach for evaluating privacy in machine learning applications. Despite their significance in identifying records belonging to the training dataset, several concerns remain unexplored, particularly with regard to tabular data. In this paper, first, we provide an extensive review and analysis of MIAs considering two main learning paradigms: centralized and federated learning. We extend and refine the taxonomy for both. Second, we demonstrate the efficacy of MIAs in tabular data using several attack strategies, also including defenses. Furthermore, in a federated learning scenario, we consider the threat posed by an outsider adversary, which is often neglected. Third, we demonstrate the high vulnerability of single-outs (records with a unique signature) to MIAs. Lastly, we explore how MIAs transfer across model architectures. Our results point towards a general poor performance of these attacks in tabular data which contrasts with previous state-of-the-art. Notably, even attacks with limited attack performance can still successfully expose a large portion of single-outs. Moreover, our findings suggest that using different surrogate models makes MIAs more effective.
Paper Structure (33 sections, 7 equations, 6 figures, 7 tables)

This paper contains 33 sections, 7 equations, 6 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Literature Review
  4. MIAs in Centralized Learning
  5. Access to target output vector
  6. Access to target model's internals
  7. Defenses to MIAs in Centralized Learning
  8. Model Hardening Defenses
  9. API Hardening Defenses
  10. MIAs in Federated Learning
  11. Update-based attacks
  12. Trend-based attacks
  13. Local shadow model-based
  14. Global shadow model-based
  15. Defenses to MIAs in Federated Learning
  16. ...and 18 more sections

Figures (6)

  • Figure 1: Taxonomy for Membership Inference Attacks conducted in centralized learning paradigm.
  • Figure 2: Taxonomy for Membership Inference Attacks conducted in federated learning paradigm.
  • Figure 3: MIAs efficacy tested on ten datasets and six machine learning models for different learning paradigms.
  • Figure 4: MIA Advantage using metric-based attacks on the centralized learning paradigm.
  • Figure 5: Comparison of MIAs efficacy on Decision Trees for centralized learning without and with defense, including hyperparameter-tuning and pruning.
  • ...and 1 more figures

Theorems & Definitions (1)

  • Definition 1: Membership Inference Game