Fair Play for Individuals, Foul Play for Groups? Auditing Anonymization's Impact on ML Fairness

TL;DR

This work addresses how privacy-preserving anonymization techniques, notably $k$-anonymity, $\ell$-diversity, and $t$-closeness, affect ML fairness and utility. It conducts a systematic audit across multiple datasets (Adult, Compas, ACSIncome) and models (including XGBoost) to measure group fairness metrics (MAD, EOD, SPD) and individual fairness metrics (LF, SF, NCF, ALF, ASF), under varying anonymization strengths, suppression levels, target distributions, and data sizes. The study finds that anonymization generally degrades group fairness while improving similarity-based individual fairness; utility declines with stronger anonymization, with non-monotonic effects and dataset/model-dependent variations. Based on these results, it offers practical guidelines for balancing privacy, fairness, and utility in anonymized ML pipelines and identifies directions for theoretical and method development to co-optimize fairness with privacy.

Abstract

Machine learning (ML) algorithms are heavily based on the availability of training data, which, depending on the domain, often includes sensitive information about data providers. This raises critical privacy concerns. Anonymization techniques have emerged as a practical solution to address these issues by generalizing features or suppressing data to make it more difficult to accurately identify individuals. Although recent studies have shown that privacy-enhancing technologies can influence ML predictions across different subgroups, thus affecting fair decision-making, the specific effects of anonymization techniques, such as $k$-anonymity, $\ell$-diversity, and $t$-closeness, on ML fairness remain largely unexplored. In this work, we systematically audit the impact of anonymization techniques on ML fairness, evaluating both individual and group fairness. Our quantitative study reveals that anonymization can degrade group fairness metrics by up to fourfold. Conversely, similarity-based individual fairness metrics tend to improve under stronger anonymization, largely as a result of increased input homogeneity. By analyzing varying levels of anonymization across diverse privacy settings and data distributions, this study provides critical insights into the trade-offs between privacy, fairness, and utility, offering actionable guidelines for responsible AI development. Our code is publicly available at: https://github.com/hharcolezi/anonymity-impact-fairness.

Figures (30)

• Figure 1: Impact of anonymity methods ($k$-anonymity, $\ell$-diversity, $t$-closeness) on group fairness metrics (MAD, EOD, SPD), individual fairness metrics (ALF, ASF, NCF), and utility metrics (Accuracy, F1-score, ROC AUC) in ML. The results are based on the Adult dataset, with gender as the protected attribute for fairness evaluation.
• Figure 2: Effect of allowed record suppression level ($\texttt{supp\_level} \in \{10, 20, 30, 40, 50\}$) in anonymization techniques ($10$-anonymity, $2$-diversity, $0.5$-closeness) on group fairness (MAD, EOD, SPD), individual fairness (ALF, ASF, NCF), and utility (Accuracy, F1-score, ROC AUC) metrics in ML. The results are derived from the Adult dataset, using gender as the protected attribute for fairness evaluation.
• Figure 3: Effect of target distribution variation (i.e., thresholded at deciles ranging from 10% to 90%) on anonymity techniques ($10$-anonymity, $2$-diversity, $0.5$-closeness) regarding group fairness (MAD, EOD, SPD), individual fairness (ALF, ASF, NCF), and utility (Accuracy, F1-score, ROC AUC) metrics in ML. Results are presented for the Adult dataset, with gender serving as the protected attribute for fairness evaluation.
• Figure 4: Effect of varying data fraction on the performance of anonymity techniques ($10$-anonymity, $2$-diversity, $0.5$-closeness) in terms of group fairness metrics (MAD, EOD, SPD), individual fairness metrics (ALF, ASF, NCF), and utility metrics (Accuracy, F1-score, ROC AUC) in ML. This analysis is performed using the Adult dataset, considering gender as the protected attribute for fairness evaluation.
• Figure 5: Comparison of the impact of different state-of-the-art ML classifiers on anonymized dataset ($k$-anonymity, $\ell$-diversity, $t$-closeness) and relation to group fairness (MAD, EOD, SPD), individual fairness (ALF, ASF, NCF), and utility (Accuracy, F1-score, ROC AUC) metrics in ML. Results are based on the Adult dataset, with gender as the protected attribute for fairness evaluation.