Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Does Differential Privacy Impact Bias in Pretrained NLP Models?

Md. Khairul Islam, Andrew Wang, Tianhao Wang, Yangfeng Ji, Judy Fox, Jieyu Zhao

TL;DR

The results show that the impact of DP on bias is not only affected by the privacy protection level but also the underlying distribution of the dataset.

Abstract

Differential privacy (DP) is applied when fine-tuning pre-trained large language models (LLMs) to limit leakage of training examples. While most DP research has focused on improving a model's privacy-utility tradeoff, some find that DP can be unfair to or biased against underrepresented groups. In this work, we show the impact of DP on bias in LLMs through empirical analysis. Differentially private training can increase the model bias against protected groups w.r.t AUC-based bias metrics. DP makes it more difficult for the model to differentiate between the positive and negative examples from the protected groups and other groups in the rest of the population. Our results also show that the impact of DP on bias is not only affected by the privacy protection level but also the underlying distribution of the dataset.

Does Differential Privacy Impact Bias in Pretrained NLP Models?

TL;DR

The results show that the impact of DP on bias is not only affected by the privacy protection level but also the underlying distribution of the dataset.

Abstract

Differential privacy (DP) is applied when fine-tuning pre-trained large language models (LLMs) to limit leakage of training examples. While most DP research has focused on improving a model's privacy-utility tradeoff, some find that DP can be unfair to or biased against underrepresented groups. In this work, we show the impact of DP on bias in LLMs through empirical analysis. Differentially private training can increase the model bias against protected groups w.r.t AUC-based bias metrics. DP makes it more difficult for the model to differentiate between the positive and negative examples from the protected groups and other groups in the rest of the population. Our results also show that the impact of DP on bias is not only affected by the privacy protection level but also the underlying distribution of the dataset.

Paper Structure

This paper contains 33 sections, 2 equations, 5 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Model Bias in NLP
  4. Terminology
  5. Protected Attributes
  6. Bias Evaluation Metrics
  7. Differential Privacy
  8. DP in machine learning:
  9. Impact of DP Methods on Fairness:
  10. Datasets
  11. Jigsaw Unintended Bias
  12. Train/Validation/Test split:
  13. UCBerkeley Hate Speech
  14. Pre-processing:
  15. Train/Validation/Test split:
  16. ...and 18 more sections

Figures (5)

  • Figure 1: AUC based bias borkan2019nuanced. BNSP for Jigsaw and BPSN for UCBerkeley drop significantly with a much smaller $\epsilon$. The larger the drop, the more biased the model w.r.t that metric.
  • Figure 2: Equality of Odds
  • Figure 3: Equality of Opportunity
  • Figure 4: Parity and Protected Accuracy
  • Figure 5: Precision Recall