Representation Magnitude has a Liability to Privacy Vulnerability

TL;DR

The paper identifies representation magnitude in bottleneck features as a key factor in privacy leakage from membership inference attacks. It proposes the Saturn Rings Classifier Module (SRCM), a plug-in that constrains representation space using a Saturn Rings Activation Function (SR) and a Magnitude Normalized Linear Layer (LinearNorm) to align training and evaluation behavior. Empirical results across CIFAR-10/100 and Purchase100 show SRCM improves privacy protection, particularly on larger models, and can further enhance defense when combined with methods like RelaxLoss, while incurring minimal inference overhead. This work provides a practical, architecture-level strategy to reduce membership leakage without sacrificing generalization, highlighting the value of regulating representation magnitude in secure ML deployment.

Abstract

The privacy-preserving approaches to machine learning (ML) models have made substantial progress in recent years. However, it is still opaque in which circumstances and conditions the model becomes privacy-vulnerable, leading to a challenge for ML models to maintain both performance and privacy. In this paper, we first explore the disparity between member and non-member data in the representation of models under common training frameworks. We identify how the representation magnitude disparity correlates with privacy vulnerability and address how this correlation impacts privacy vulnerability. Based on the observations, we propose Saturn Ring Classifier Module (SRCM), a plug-in model-level solution to mitigate membership privacy leakage. Through a confined yet effective representation space, our approach ameliorates models' privacy vulnerability while maintaining generalizability. The code of this work can be found here: \url{https://github.com/JEKimLab/AIES2024_SRCM}

Figures (8)

• Figure 1: Relationship between the distance to the origin, the distance to the decision boundary, and MIAs accuracy. For a sample's distance to the decision boundary, we use the difference between 1st and 2nd maximum prediction probabilities as the metric. The four charts are obtained by averaging the results of dozens of independent experiments. The charts in blue ((a) & (c)) are produced on the training set, and the other charts in green ((b) & (d)) are produced on the testing set. (ResNet18, CIFAR-100)
• Figure 2: Illustration of Saturn rings activation function in 2D vector space. Our proposed method pushes samples into the representation space.
• Figure 3: SCRM and variational designs for proof-of-concept purposes. $\rightarrow$ denotes common phase, $\rightarrow$ denotes evaluation phase, and $\rightarrow$ denotes training phase.
• Figure 4: Comparison of our proposed SRCM and two other variational designing options for proof of concept - one without LinearNorm, and the other with a Linear layer in the evaluation phase. 'Vanilla' denotes the original model (baseline). Rightward (higher testing accuracy) and lower (lowe MIA accuracy) is better. (ResNet18, CIFAR-100)
• Figure 5: Training, testing, and MIAs accuracy changes with various hyper-parameters combinations using SRCM. (ResNet18, CIFAR-100)