Membership Inference Test: Auditing Training Data in Object Classification Models

TL;DR

This research analyzed the performance of Membership Inference Tests (MINT), focusing on determining whether given data were utilized during the training phase, specifically in the domain of object recognition, and proposed and developed architectures tailored for MINT models.

Abstract

In this research, we analyze the performance of Membership Inference Tests (MINT), focusing on determining whether given data were utilized during the training phase, specifically in the domain of object recognition. Within the area of object recognition, we propose and develop architectures tailored for MINT models. These architectures aim to optimize performance and efficiency in data utilization, offering a tailored solution to tackle the complexities inherent in the object recognition domain. We conducted experiments involving an object detection model, an embedding extractor, and a MINT module. These experiments were performed in three public databases, totaling over 174K images. The proposed architecture leverages convolutional layers to capture and model the activation patterns present in the data during the training process. Through our analysis, we are able to identify given data used for testing and training, achieving precision rates ranging between 70% and 80%, contingent upon the depth of the detection module layer chosen for input to the MINT module. Additionally, our studies entail an analysis of the factors influencing the MINT Module, delving into the contributing elements behind more transparent training processes.

Figures (5)

• Figure 1: The objective of MINT is to discern whether a given data point (d) was utilized in the training process of an audited AI Model (M) trained with a specific database ($\mathcal{D}$).
• Figure 2: The Membership Inference Test (MINT) Model ($\mathcal{T}$) is trained to ascertain whether a given data instance ($d$) was included in the training dataset $\mathcal{D}$ utilized by an Audited Artificial Intelligence Model (M). The inputs for the MINT Model comprise Auxiliary Auditable Data, such as activation maps corresponding to data samples $d$, as well as the predictions made by model M.
• Figure 3: MINT ROC curves evaluated over CIFAR-10 for different epochs and layers.
• Figure 4: MINT ROC curves evaluated for different layers over CIFAR-100.
• Figure 5: MINT ROC curves evaluated for the 10 classes of CIFAR-10.