Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluating AI Group Fairness: a Fuzzy Logic Perspective

Emmanouil Krasanakis, Symeon Papadopoulos

TL;DR

The paper addresses the challenge of defining and evaluating group fairness in AI when context and relaxation choices are uncertain. It introduces a Basic fuzzy Logic (BL) framework that encodes fairness predicates as continuous truth values in $[0,1]$, with the logic subclass, discrimination evaluation, and group-membership truth values driven by stakeholder beliefs. By standardizing bias and fairness expressions and deriving their closed-form BL formulations, the approach enables context-aware, interpretable, and transferable fairness definitions, demonstrated across replicating discrimination measures, multidimensional fairness, ABROCA adaptations, and Hooker-Williams criteria. This framework supports interdisciplinary collaboration and offers a flexible, mathematically precise way to reinterpret and extend existing non-probabilistic fairness definitions in diverse social contexts.

Abstract

Artificial intelligence systems often address fairness concerns by evaluating and mitigating measures of group discrimination, for example that indicate biases against certain genders or races. However, what constitutes group fairness depends on who is asked and the social context, whereas definitions are often relaxed to accept small deviations from the statistical constraints they set out to impose. Here we decouple definitions of group fairness both from the context and from relaxation-related uncertainty by expressing them in the axiomatic system of Basic fuzzy Logic (BL) with loosely understood predicates, like encountering group members. We then evaluate the definitions in subclasses of BL, such as Product or Lukasiewicz logics. Evaluation produces continuous instead of binary truth values by choosing the logic subclass and truth values for predicates that reflect uncertain context-specific beliefs, such as stakeholder opinions gathered through questionnaires. Internally, it follows logic-specific rules to compute the truth values of definitions. We show that commonly held propositions standardize the resulting mathematical formulas and we transcribe logic and truth value choices to layperson terms, so that anyone can answer them. We also use our framework to study several literature definitions of algorithmic fairness, for which we rationalize previous expedient practices that are non-probabilistic and show how to re-interpret their formulas and parameters in new contexts.

Evaluating AI Group Fairness: a Fuzzy Logic Perspective

TL;DR

The paper addresses the challenge of defining and evaluating group fairness in AI when context and relaxation choices are uncertain. It introduces a Basic fuzzy Logic (BL) framework that encodes fairness predicates as continuous truth values in , with the logic subclass, discrimination evaluation, and group-membership truth values driven by stakeholder beliefs. By standardizing bias and fairness expressions and deriving their closed-form BL formulations, the approach enables context-aware, interpretable, and transferable fairness definitions, demonstrated across replicating discrimination measures, multidimensional fairness, ABROCA adaptations, and Hooker-Williams criteria. This framework supports interdisciplinary collaboration and offers a flexible, mathematically precise way to reinterpret and extend existing non-probabilistic fairness definitions in diverse social contexts.

Abstract

Artificial intelligence systems often address fairness concerns by evaluating and mitigating measures of group discrimination, for example that indicate biases against certain genders or races. However, what constitutes group fairness depends on who is asked and the social context, whereas definitions are often relaxed to accept small deviations from the statistical constraints they set out to impose. Here we decouple definitions of group fairness both from the context and from relaxation-related uncertainty by expressing them in the axiomatic system of Basic fuzzy Logic (BL) with loosely understood predicates, like encountering group members. We then evaluate the definitions in subclasses of BL, such as Product or Lukasiewicz logics. Evaluation produces continuous instead of binary truth values by choosing the logic subclass and truth values for predicates that reflect uncertain context-specific beliefs, such as stakeholder opinions gathered through questionnaires. Internally, it follows logic-specific rules to compute the truth values of definitions. We show that commonly held propositions standardize the resulting mathematical formulas and we transcribe logic and truth value choices to layperson terms, so that anyone can answer them. We also use our framework to study several literature definitions of algorithmic fairness, for which we rationalize previous expedient practices that are non-probabilistic and show how to re-interpret their formulas and parameters in new contexts.
Paper Structure (24 sections, 7 theorems, 52 equations, 7 figures, 3 tables)

This paper contains 24 sections, 7 theorems, 52 equations, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Related work
  4. Fuzzy Logic and BL
  5. Evaluating Group Fairness in Fuzzy Logic
  6. Why not stick to probabilities?
  7. Terminology and propositions
  8. Evaluating group fairness
  9. Truth values from stakeholder beliefs
  10. Case Studies with Practical Results
  11. Replicating base measures of discrimination
  12. Aggregating multidimensional discrimination
  13. ABROCA in Product logic
  14. On the Hooker-Williams fairness criterion
  15. Discussion
  16. ...and 9 more sections

Key Result

Proposition 1

Group imbalance $Imb$ is an implication from group membership to some property such that $\mathop{\mathrm{\mathcal{P}}}\nolimits(Imb)=1$ for no group members and $Imb\to E$ otherwise.

Figures (7)

  • Figure 1: Our fuzzy framework for evaluating fairness. Shaded cells denote inputs/outputs.
  • Figure 2: Relation between terms employed in our fairness propositions.
  • Figure 3: Two pairs of different sensitive and non-sensitive attribute ROCs with similar ABROCA but different RBROCA.
  • Figure 4: Contour of the truth value of HW (left) and FairHW (right), across the utility values of two individuals (axes), and for $\delta=0.2$.
  • Figure 5: Interdisciplinary collaboration to apply our framework in practice.
  • ...and 2 more figures

Theorems & Definitions (10)

  • Proposition 1
  • Proposition 2
  • Proposition 3
  • Proposition 4: optional
  • Theorem 1
  • Theorem 2
  • Theorem 3
  • proof
  • proof
  • proof