Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An investigation of structures responsible for gender bias in BERT and DistilBERT

Thibaud Leteno, Antoine Gourru, Charlotte Laclau, Christophe Gravier

TL;DR

This work tackles gender bias in large transformer models by proposing an empirical neural-exploration protocol that jointly analyzes BERT and DistilBERT under balanced and imbalanced fine-tuning. By evaluating attention and hidden-state dynamics (E1) and conducting head ablations (E2) on the Bias in Bios Curriculum Vitae dataset, the study shows that bias is not confined to a single layer or head, but distributes across the architecture, with the last layers displaying stronger divergence during fine-tuning. Importantly, DistilBERT demonstrates more homogeneous head representations and greater robustness to double imbalance, suggesting it as a preferable option in scenarios with underrepresented classes and sensitive attributes. The findings highlight the danger of indiscriminate head removal and provide a practical protocol for diagnosing and mitigating bias in compressed and uncompressed transformers, with implications for fair deployment in real-world NLP systems.

Abstract

In recent years, large Transformer-based Pre-trained Language Models (PLM) have changed the Natural Language Processing (NLP) landscape, by pushing the performance boundaries of the state-of-the-art on a wide variety of tasks. However, this performance gain goes along with an increase in complexity, and as a result, the size of such models (up to billions of parameters) represents a constraint for their deployment on embedded devices or short-inference time tasks. To cope with this situation, compressed models emerged (e.g. DistilBERT), democratizing their usage in a growing number of applications that impact our daily lives. A crucial issue is the fairness of the predictions made by both PLMs and their distilled counterparts. In this paper, we propose an empirical exploration of this problem by formalizing two questions: (1) Can we identify the neural mechanism(s) responsible for gender bias in BERT (and by extension DistilBERT)? (2) Does distillation tend to accentuate or mitigate gender bias (e.g. is DistilBERT more prone to gender bias than its uncompressed version, BERT)? Our findings are the following: (I) one cannot identify a specific layer that produces bias; (II) every attention head uniformly encodes bias; except in the context of underrepresented classes with a high imbalance of the sensitive attribute; (III) this subset of heads is different as we re-fine tune the network; (IV) bias is more homogeneously produced by the heads in the distilled model.

An investigation of structures responsible for gender bias in BERT and DistilBERT

TL;DR

This work tackles gender bias in large transformer models by proposing an empirical neural-exploration protocol that jointly analyzes BERT and DistilBERT under balanced and imbalanced fine-tuning. By evaluating attention and hidden-state dynamics (E1) and conducting head ablations (E2) on the Bias in Bios Curriculum Vitae dataset, the study shows that bias is not confined to a single layer or head, but distributes across the architecture, with the last layers displaying stronger divergence during fine-tuning. Importantly, DistilBERT demonstrates more homogeneous head representations and greater robustness to double imbalance, suggesting it as a preferable option in scenarios with underrepresented classes and sensitive attributes. The findings highlight the danger of indiscriminate head removal and provide a practical protocol for diagnosing and mitigating bias in compressed and uncompressed transformers, with implications for fair deployment in real-world NLP systems.

Abstract

In recent years, large Transformer-based Pre-trained Language Models (PLM) have changed the Natural Language Processing (NLP) landscape, by pushing the performance boundaries of the state-of-the-art on a wide variety of tasks. However, this performance gain goes along with an increase in complexity, and as a result, the size of such models (up to billions of parameters) represents a constraint for their deployment on embedded devices or short-inference time tasks. To cope with this situation, compressed models emerged (e.g. DistilBERT), democratizing their usage in a growing number of applications that impact our daily lives. A crucial issue is the fairness of the predictions made by both PLMs and their distilled counterparts. In this paper, we propose an empirical exploration of this problem by formalizing two questions: (1) Can we identify the neural mechanism(s) responsible for gender bias in BERT (and by extension DistilBERT)? (2) Does distillation tend to accentuate or mitigate gender bias (e.g. is DistilBERT more prone to gender bias than its uncompressed version, BERT)? Our findings are the following: (I) one cannot identify a specific layer that produces bias; (II) every attention head uniformly encodes bias; except in the context of underrepresented classes with a high imbalance of the sensitive attribute; (III) this subset of heads is different as we re-fine tune the network; (IV) bias is more homogeneously produced by the heads in the distilled model.
Paper Structure (18 sections, 4 equations, 4 figures)

This paper contains 18 sections, 4 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Related Works
  3. Contribution
  4. Organisation
  5. Preliminaries and Background
  6. A gender-bias neural exploration protocol for language models
  7. Fine-tuning Scenarios for Fairness Evaluation
  8. Attention and Hidden States comparison (E1)
  9. Attention values comparison.
  10. Hidden states comparison.
  11. Head's ablation (E2)
  12. Performance.
  13. Gender fairness.
  14. Experiments
  15. Task and Dataset
  16. ...and 3 more sections

Figures (4)

  • Figure 1: Scheme of head's noising experiments
  • Figure 2: JS divergence comparison for BERT (left) and DistilBERT (center-left) and SVCCA distance for BERT (center-right) and DistilBERT (right).
  • Figure 3: (a) F-Score and (b) Equalized Odds for model Mi per class, without ablation and ordered by ascending ratio W/M.
  • Figure 4: EO amplitude for BERT's Mi (a), DistilBERT's Mi (b), BERT's and DistilBERT's Mb (c).