On the Importance of Difficulty Calibration in Membership Inference Attacks

TL;DR

This paper tackles the instability of membership inference attacks caused by high false positive rates. It introduces difficulty calibration, a post-processing step that adjusts membership scores by comparing to typical models trained on the same data distribution, substantially reducing false positives without hurting accuracy. Empirical results across multiple datasets show calibrated attacks achieve higher AUC and improved precision-recall trade-offs, with gains up to about 0.1 in AUC, even under data augmentation. It also connects calibration via forgetting to efficient white-box attacks and discusses implications for privacy risk evaluation and future attack design.

Abstract

The vulnerability of machine learning models to membership inference attacks has received much attention in recent years. However, existing attacks mostly remain impractical due to having high false positive rates, where non-member samples are often erroneously predicted as members. This type of error makes the predicted membership signal unreliable, especially since most samples are non-members in real world applications. In this work, we argue that membership inference attacks can benefit drastically from \emph{difficulty calibration}, where an attack's predicted membership score is adjusted to the difficulty of correctly classifying the target sample. We show that difficulty calibration can significantly reduce the false positive rate of a variety of existing attacks without a loss in accuracy.

Figures (9)

• Figure 1: Histogram of the negative loss score (cf.\ref{['eq:loss_score']}) before and after difficulty calibration. Without calibration, the member and non-member scores overlap significantly, and it is impossible to determine a threshold that results in low FPR. After calibration, the highest scored samples mostly belong to the member class, enabling high precision and low FPR attacks.
• Figure 2: Left: ROC (left) and precision-recall (right) curves of calibrated/uncalibrated loss score attacks on CIFAR10. The threshold $\tau$ that optimizes accuracy is shown as a red dot. Calibration yields a higher TPR for the same value of FPR, or equivalently a higher precision at low levels of recall. The precision-recall trade-off surfaces very different behaviors for two methods that have otherwise very similar accuracy.
• Figure 3: AUC (left) and accuracy (right) of the gap attack and calibrated/uncalibrated loss score attacks against target model trained on varying training set sizes on CIFAR10. Both AUC and accuracy increase as the training dataset size decreases due to more severe overfitting of the target model. Difficulty calibration can effectively leverage this to improve the attack's AUC and accuracy across all training set sizes.
• Figure 4: Accuracy (left) and precision-recall curve (right) of the loss score attack when changing the member to non-member ratio. At low ratios, the calibrated attack can still identify members with high precision, whereas the uncalibrated attack is unable to do so despite high accuracy.
• Figure 5: AUC (left) and attack accuracy (right) for the calibrated loss attack with varying number of reference models on CIFAR-10. Using more reference models and training reference models with a higher shadow dataset size strictly improve performance at a cost of more computation.