Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Explanations as Bias Detectors: A Critical Study of Local Post-hoc XAI Methods for Fairness Exploration

Vasiliki Papanikou, Danae Pla Karidi, Evaggelia Pitoura, Emmanouil Panagiotou, Eirini Ntoutsi

TL;DR

This work addresses how local post-hoc explanations can serve as bias detectors for fairness exploration in AI. It introduces a pipeline that computes explanations for individuals, aggregates them by demographic groups, and compares the results against distributive fairness metrics. Through extensive experiments on diverse datasets (e.g., Adult variants and COMPAS) and models (Random Forest and XGBoost), the study shows that explanation signals align with observed fairness gaps and that removing protected attributes shifts bias toward proxy features, though not eliminating disparities. The findings underscore the potential and challenges of using XAI for fairness auditing, highlighting the need for careful method choice, aggregation, and validation to support trustworthy decision support.

Abstract

As Artificial Intelligence (AI) is increasingly used in areas that significantly impact human lives, concerns about fairness and transparency have grown, especially regarding their impact on protected groups. Recently, the intersection of explainability and fairness has emerged as an important area to promote responsible AI systems. This paper explores how explainability methods can be leveraged to detect and interpret unfairness. We propose a pipeline that integrates local post-hoc explanation methods to derive fairness-related insights. During the pipeline design, we identify and address critical questions arising from the use of explanations as bias detectors such as the relationship between distributive and procedural fairness, the effect of removing the protected attribute, the consistency and quality of results across different explanation methods, the impact of various aggregation strategies of local explanations on group fairness evaluations, and the overall trustworthiness of explanations as bias detectors. Our results show the potential of explanation methods used for fairness while highlighting the need to carefully consider the aforementioned critical aspects.

Explanations as Bias Detectors: A Critical Study of Local Post-hoc XAI Methods for Fairness Exploration

TL;DR

This work addresses how local post-hoc explanations can serve as bias detectors for fairness exploration in AI. It introduces a pipeline that computes explanations for individuals, aggregates them by demographic groups, and compares the results against distributive fairness metrics. Through extensive experiments on diverse datasets (e.g., Adult variants and COMPAS) and models (Random Forest and XGBoost), the study shows that explanation signals align with observed fairness gaps and that removing protected attributes shifts bias toward proxy features, though not eliminating disparities. The findings underscore the potential and challenges of using XAI for fairness auditing, highlighting the need for careful method choice, aggregation, and validation to support trustworthy decision support.

Abstract

As Artificial Intelligence (AI) is increasingly used in areas that significantly impact human lives, concerns about fairness and transparency have grown, especially regarding their impact on protected groups. Recently, the intersection of explainability and fairness has emerged as an important area to promote responsible AI systems. This paper explores how explainability methods can be leveraged to detect and interpret unfairness. We propose a pipeline that integrates local post-hoc explanation methods to derive fairness-related insights. During the pipeline design, we identify and address critical questions arising from the use of explanations as bias detectors such as the relationship between distributive and procedural fairness, the effect of removing the protected attribute, the consistency and quality of results across different explanation methods, the impact of various aggregation strategies of local explanations on group fairness evaluations, and the overall trustworthiness of explanations as bias detectors. Our results show the potential of explanation methods used for fairness while highlighting the need to carefully consider the aforementioned critical aspects.
Paper Structure (18 sections, 3 equations, 22 figures, 9 tables)

This paper contains 18 sections, 3 equations, 22 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Motivation and Methodology
  4. Experimental Evaluation
  5. Experimental setup
  6. Results for RQ1: How does feature attribution relate to distributive fairness? What is the relationship, if any, between process fairness and output fairness?
  7. Results for RQ2: What is the impact of removing protected attributes on fairness, and how does it relate to direct vs. indirect discrimination
  8. Results for RQ3: Can aggregated individual/local explanations provide global fairness insights?
  9. Related Work
  10. Conclusion and Future Work
  11. Appendix
  12. Datasets
  13. RQ1. How does feature attribution relate to statistical group fairness? What is the relationship, if any, between process fairness and output fairness?
  14. RQ2. What is the effect in fairness of removing the protected attribute? How this relates to direct (dependency on the protected attribute) vs indirect discrimination (dependency on proxy attributes)?
  15. Generation of correlation matrices
  16. ...and 3 more sections

Figures (22)

  • Figure 1: An overview of the proposed pipeline integrating local post-hoc explanations and addressing key design/research questions for deriving fairness insights.
  • Figure 2: LIME and SHAP feature contributions for sex and race across datasets.
  • Figure 3: LIME mean feature contributions for sex and race across datasets.
  • Figure 4: Mean contributions for features in the Adult, AdultCA, and AdultLA datasets.
  • Figure 5: Percentage of feature changes from the DiCE method per group for the Adult, AdultCA, and AdultLA datasets.
  • ...and 17 more figures