Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learnability and Privacy Vulnerability are Entangled in a Few Critical Weights

Xingli Fang, Jung-Eun Kim

Abstract

Prior approaches for membership privacy preservation usually update or retrain all weights in neural networks, which is costly and can lead to unnecessary utility loss or even more serious misalignment in predictions between training data and non-training data. In this work, we observed three insights: i) privacy vulnerability exists in a very small fraction of weights; ii) however, most of those weights also critically impact utility performance; iii) the importance of weights stems from their locations rather than their values. According to these insights, to preserve privacy, we score critical weights, and instead of discarding those neurons, we rewind only the weights for fine-tuning. We show that, through extensive experiments, this mechanism exhibits outperforming resilience in most cases against Membership Inference Attacks while maintaining utility.

Learnability and Privacy Vulnerability are Entangled in a Few Critical Weights

Abstract

Prior approaches for membership privacy preservation usually update or retrain all weights in neural networks, which is costly and can lead to unnecessary utility loss or even more serious misalignment in predictions between training data and non-training data. In this work, we observed three insights: i) privacy vulnerability exists in a very small fraction of weights; ii) however, most of those weights also critically impact utility performance; iii) the importance of weights stems from their locations rather than their values. According to these insights, to preserve privacy, we score critical weights, and instead of discarding those neurons, we rewind only the weights for fine-tuning. We show that, through extensive experiments, this mechanism exhibits outperforming resilience in most cases against Membership Inference Attacks while maintaining utility.
Paper Structure (31 sections, 7 equations, 10 figures, 4 tables, 1 algorithm)

This paper contains 31 sections, 7 equations, 10 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries and Related Work (more continued in Appendix)
  3. Introduction to Membership Inference Attacks
  4. Importance Estimation of components in neural networks
  5. Motivation: Removing Unimportant Weights is Ineffective for Privacy
  6. Problem Setup and Methodology
  7. Privacy Vulnerability Estimation
  8. Learnability and Privacy Vulnerability are Entangled
  9. Critical Weights Rewinding and Finetuning (CWRF)
  10. Weights Rewinding.
  11. Weights Freezing & Privacy Fine-Tuning.
  12. The Privacy-Vulnerable Weights are Unnecessary to be Trained
  13. Empirical Study
  14. Experimental Setups
  15. Datasets.
  16. ...and 16 more sections

Figures (10)

  • Figure 1: According to TFO, important weights are pruned over different sparsities. The results are shown on ResNet18 and CIFAR-100
  • Figure 2: Our approach takes into account privacy vulnerability for importance estimation, while TFO only measures learnability for accuracy.
  • Figure 3: The visualization of weight-level learnability scores and privacy vulnerability scores. Privacy vulnerability and accuracy are significantly correlated and this correlation varies in different components. Due to the significant scale discrepancy, the ranges of axes of the four charts in ViT are not consistent. (The same data points as Tab.\ref{['tab:learning_privacy_pcc']}.)
  • Figure 4: The performance of M1, M2, & M3 on ResNet18 & CIFAR-100.
  • Figure 5: The performance of A1, A2, & A3 along with removing/rewinding ratios. The dotted line represents a baseline performance of a model trained from scratch with the same privacy-preserving approach
  • ...and 5 more figures