CLMIA: Membership Inference Attacks via Unsupervised Contrastive Learning

TL;DR

This paper proposes a new attack method called CLMIA, which uses unsupervised contrastive learning to train an attack model without using extra membership status information, and demonstrates that the attack method performs better with less labeled identity information, which applies to more realistic scenarios.

Abstract

Since machine learning model is often trained on a limited data set, the model is trained multiple times on the same data sample, which causes the model to memorize most of the training set data. Membership Inference Attacks (MIAs) exploit this feature to determine whether a data sample is used for training a machine learning model. However, in realistic scenarios, it is difficult for the adversary to obtain enough qualified samples that mark accurate identity information, especially since most samples are non-members in real world applications. To address this limitation, in this paper, we propose a new attack method called CLMIA, which uses unsupervised contrastive learning to train an attack model without using extra membership status information. Meanwhile, in CLMIA, we require only a small amount of data with known membership status to fine-tune the attack model. Experimental results demonstrate that CLMIA performs better than existing attack methods for different datasets and model structures, especially with data with less marked identity information. In addition, we experimentally find that the attack performs differently for different proportions of labeled identity information for member and non-member data. More analysis proves that our attack method performs better with less labeled identity information, which applies to more realistic scenarios.

Figures (8)

• Figure 1: (a) Maximum output value of the last layer of the target model. The first 5000 samples are members, and the last 5000 are non-members. (b) Information entropy of the posterior probabilities of the target model.
• Figure 2: System Model of Our CLMIA
• Figure 3: ROC curves for attacks on three different datasets and three model architectures (from top to bottom: Simple CNN, Resnet-18, VGG-19)
• Figure 4: The effect of the size of the labeled dataset on the balanced accuracy in the CIFAR-100 dataset, with a model structure of Resnet-18.
• Figure 5: The performance of CLMIA and Only FC layer on the test dataset during training. Where (a) denotes performance on dataset CIFAR-100 and model structure Resnet-18. (b) denotes performance on dataset CIFAR-100 and model structure VGG-19.