A Critical Review of Predominant Bias in Neural Networks

TL;DR

This work aims to restore clarity by providing two mathematical definitions for these two predominant biases and leveraging these definitions to unify a comprehensive list of papers in order to alleviate confusion regarding these two types of biases.

Abstract

Bias issues of neural networks garner significant attention along with its promising advancement. Among various bias issues, mitigating two predominant biases is crucial in advancing fair and trustworthy AI: (1) ensuring neural networks yields even performance across demographic groups, and (2) ensuring algorithmic decision-making does not rely on protected attributes. However, upon the investigation of \pc papers in the relevant literature, we find that there exists a persistent, extensive but under-explored confusion regarding these two types of biases. Furthermore, the confusion has already significantly hampered the clarity of the community and subsequent development of debiasing methodologies. Thus, in this work, we aim to restore clarity by providing two mathematical definitions for these two predominant biases and leveraging these definitions to unify a comprehensive list of papers. Next, we highlight the common phenomena and the possible reasons for the existing confusion. To alleviate the confusion, we provide extensive experiments on synthetic, census, and image datasets, to validate the distinct nature of these biases, distinguish their different real-world manifestations, and evaluate the effectiveness of a comprehensive list of bias assessment metrics in assessing the mitigation of these biases. Further, we compare these two types of biases from multiple dimensions including the underlying causes, debiasing methods, evaluation protocol, prevalent datasets, and future directions. Last, we provide several suggestions aiming to guide researchers engaged in bias-related work to avoid confusion and further enhance clarity in the community.

Figures (10)

• Figure 1: The same set of terminology about bias is interpreted differently by experts, which significantly confuses the understanding of the audience. By investigating 415 papers about prevalent bias issues, we discover that there exists significant confusion regarding these prevalent bias issues. The confusion is evident in several ways such as ambiguity of terminology, inaccurate motivation, and lack of terminology reuse. Most notably, several studies inaccurately motivate themselves on a particular bias while actually addressing a different type of bias. This prevailing confusion considerably impedes the clarity of related work. Thus, we propose new definitions to unify the existing literature and pave a clear path for future research.
• Figure 2: The enrichment of the concept "bias" in machine intelligence with important milestones. Initially, "bias" implied that human decision-making depends on protected attributes (Type II Bias). As machine intelligence began aiding human decision-making processes, the subject of "bias" broadened from humans to algorithms. Along with the continued advances of machine intelligence, a new aspect of bias issues, performance disparity across demographic groups (Type I Bias), further enriched the meaning of "bias". Currently, addressing both Type I Bias and Type II Bias becomes essential for ensuring Trustworthy AI.
• Figure 3: Distribution of training and testing sets regarding synthetic data. The vertical classification boundary (labeled as the black line) reveals that the classifier does not utilize $A$ for classification. However, there are more wrong predictions in the group of $A=-1$ than in the group of $A=1$, which violates performance parity.
• Figure 4: Type I Bias exists without Type II Bias since there exists accuracy disparity across $A$ while $\hat{Y}$ and $A$ are independent.
• Figure 5: Distribution of training and testing sets regarding synthetic data. The non-vertical classification boundary (labeled as the black line) reveals that the classifier utilizes $A$ for classification. However, the number of wrong predictions is approximately the same across $A$, thereby fulfilling performance parity.

Theorems & Definitions (4)

• Definition 1
• Definition 2
• Definition 3
• Definition 4