Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Impact of Generalization Techniques on the Interplay Among Privacy, Utility, and Fairness in Image Classification

Ahmad Hassanpour, Amir Zarei, Khawla Mallat, Anderson Santana de Oliveira, Bian Yang

TL;DR

The work investigates how generalization techniques interact with privacy and fairness in image classification under differential privacy. By evaluating DP-SAT alongside GN, OBS, WS, AM, and PA across CIFAR-10/100, synthetic biases (CIFAR-10S/100S), and CelebA, it demonstrates that DP-SAT can improve private accuracy (e.g., 81.11% on CIFAR-10 under $(8,10^{-5})$-DP) and generally enhances the privacy-utility balance compared to DP-SGD. However, the same techniques tend to amplify bias on biased data and real-world attributes, with higher MIA AUC and worsened HS in several settings; the Onion Effect further reveals persistent privacy vulnerabilities as outliers are removed. To address these trade-offs, the authors introduce Harmonic Score (HS) to jointly gauge accuracy, privacy leakage, and fairness, and validate findings in CelebA, highlighting practical implications for designing privacy-preserving, fair image classifiers. Overall, the work clarifies the promises and limits of generalization techniques in private learning and provides a roadmap for balancing competing objectives in real-world datasets.

Abstract

This study investigates the trade-offs between fairness, privacy, and utility in image classification using machine learning (ML). Recent research suggests that generalization techniques can improve the balance between privacy and utility. One focus of this work is sharpness-aware training (SAT) and its integration with differential privacy (DP-SAT) to further improve this balance. Additionally, we examine fairness in both private and non-private learning models trained on datasets with synthetic and real-world biases. We also measure the privacy risks involved in these scenarios by performing membership inference attacks (MIAs) and explore the consequences of eliminating high-privacy risk samples, termed outliers. Moreover, we introduce a new metric, named \emph{harmonic score}, which combines accuracy, privacy, and fairness into a single measure. Through empirical analysis using generalization techniques, we achieve an accuracy of 81.11\% under $(8, 10^{-5})$-DP on CIFAR-10, surpassing the 79.5\% reported by De et al. (2022). Moreover, our experiments show that memorization of training samples can begin before the overfitting point, and generalization techniques do not guarantee the prevention of this memorization. Our analysis of synthetic biases shows that generalization techniques can amplify model bias in both private and non-private models. Additionally, our results indicate that increased bias in training data leads to reduced accuracy, greater vulnerability to privacy attacks, and higher model bias. We validate these findings with the CelebA dataset, demonstrating that similar trends persist with real-world attribute imbalances. Finally, our experiments show that removing outlier data decreases accuracy and further amplifies model bias.

The Impact of Generalization Techniques on the Interplay Among Privacy, Utility, and Fairness in Image Classification

TL;DR

The work investigates how generalization techniques interact with privacy and fairness in image classification under differential privacy. By evaluating DP-SAT alongside GN, OBS, WS, AM, and PA across CIFAR-10/100, synthetic biases (CIFAR-10S/100S), and CelebA, it demonstrates that DP-SAT can improve private accuracy (e.g., 81.11% on CIFAR-10 under -DP) and generally enhances the privacy-utility balance compared to DP-SGD. However, the same techniques tend to amplify bias on biased data and real-world attributes, with higher MIA AUC and worsened HS in several settings; the Onion Effect further reveals persistent privacy vulnerabilities as outliers are removed. To address these trade-offs, the authors introduce Harmonic Score (HS) to jointly gauge accuracy, privacy leakage, and fairness, and validate findings in CelebA, highlighting practical implications for designing privacy-preserving, fair image classifiers. Overall, the work clarifies the promises and limits of generalization techniques in private learning and provides a roadmap for balancing competing objectives in real-world datasets.

Abstract

This study investigates the trade-offs between fairness, privacy, and utility in image classification using machine learning (ML). Recent research suggests that generalization techniques can improve the balance between privacy and utility. One focus of this work is sharpness-aware training (SAT) and its integration with differential privacy (DP-SAT) to further improve this balance. Additionally, we examine fairness in both private and non-private learning models trained on datasets with synthetic and real-world biases. We also measure the privacy risks involved in these scenarios by performing membership inference attacks (MIAs) and explore the consequences of eliminating high-privacy risk samples, termed outliers. Moreover, we introduce a new metric, named \emph{harmonic score}, which combines accuracy, privacy, and fairness into a single measure. Through empirical analysis using generalization techniques, we achieve an accuracy of 81.11\% under -DP on CIFAR-10, surpassing the 79.5\% reported by De et al. (2022). Moreover, our experiments show that memorization of training samples can begin before the overfitting point, and generalization techniques do not guarantee the prevention of this memorization. Our analysis of synthetic biases shows that generalization techniques can amplify model bias in both private and non-private models. Additionally, our results indicate that increased bias in training data leads to reduced accuracy, greater vulnerability to privacy attacks, and higher model bias. We validate these findings with the CelebA dataset, demonstrating that similar trends persist with real-world attribute imbalances. Finally, our experiments show that removing outlier data decreases accuracy and further amplifies model bias.

Paper Structure

This paper contains 23 sections, 2 equations, 9 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Differential Privacy (DP)
  4. Generalization Techniques
  5. Membership Inference Attacks (MIAs)
  6. Methodology and Experimental Setup
  7. Datasets
  8. Metrics
  9. Implementation Details
  10. Experimental Results
  11. Privacy-Utility Trade-off
  12. Detailed Analysis of the Impact of Generalization Techniques on Privacy Leakage
  13. The Comparison of DP-SGD and DP-SAT
  14. Fairness-Utility Trade-off
  15. Privacy and Fairness-Utility Trade-offs
  16. ...and 8 more sections

Figures (9)

  • Figure 1: Variations in accuracy, MIA AUC, and model bias for four datasets CIFAR-10, CIFAR-100, CIFAR-10S, and CIFAR-100S in non-private and private (i.e., $(8, 10^{-5})$-DP) learning settings. DP-CIFAR-10/10S/100/100S is used to denote when DP is applied. (a) illustrates the impact of generalization techniques (i.e., BL: baseline, OBS: optimal batch size, GN: group normalization, WS: weight standardization, AM: augmentation multiplicity, PA: parameter averaging, SAT: sharpness-aware training) on accuracy when the training data is unbiased while (b) measures such an impact on model accuracy and bias when training data is biased. (c) and (d) show MIA AUC of two threshold-based MIAs (i.e., threshold, threshold entropy) and two shadow-based MIAs (MLP: multilayer perceptron, RF: random forest). (e) compares the HS to represent the balance between accuracy, MIA AUC, and bias.
  • Figure 2: The training and test accuracy for four training processes are depicted for the CIFAR-10 dataset. We track MIA AUC (AUC) and membership probability (MP) for a specific training sample (airplane image shown in the upper left corner of each plot) for a baseline model and after adding generalization techniques (GT). Early stopping (ES) and overfitting (OF) points in the non-private training process are indicated.
  • Figure 3: Examined the impact of different privacy budgets ($\epsilon$ = 1, 2, 4, 8) and data bias (95%, 75%) on (a) accuracy, (b) MIA AUC, (c) bias. DP-CIFAR-10 and DP-CIFAR-10S are used to denote when DP is applied.
  • Figure 4: Examined the impact of different privacy budgets ($\epsilon$ = 1, 2, 4, 8) and data bias (95%, 75%) on HS.
  • Figure 5: The impact of removing outliers on model performance (a) accuracy, (b) MIA AUC, and (c) bias trained with and without DP on CIFAR-10 and CIFAR10S. DP-CIFAR-10 and DP-CIFAR-10S are used to denote when DP is applied.
  • ...and 4 more figures

Theorems & Definitions (1)

  • definition 1