Interpretable and Fair Mechanisms for Abstaining Classifiers

TL;DR

This paper addresses the challenge of producing fair decisions when using abstaining classifiers. It introduces IFAC, an interpretable abstaining mechanism that jointly accounts for uncertainty and unfairness by leveraging globally discovered discriminatory patterns and local Situation Testing, with explanations designed for human review. The approach calibrates dual rejection thresholds to control coverage while mitigating demographic disparities on non-rejected instances. Empirical results on ACSIncome and Wisconsin Recidivism show IFAC achieving competitive accuracy and reduced fairness gaps compared to standard abstention baselines, highlighting the value of human-in-the-loop oversight in high-stakes AI applications. Overall, IFAC advances transparent, regulation-aligned decision-making by making unfair rejections interpretable and actionable for human decision-makers.

Abstract

Abstaining classifiers have the option to refrain from providing a prediction for instances that are difficult to classify. The abstention mechanism is designed to trade off the classifier's performance on the accepted data while ensuring a minimum number of predictions. In this setting, often fairness concerns arise when the abstention mechanism solely reduces errors for the majority groups of the data, resulting in increased performance differences across demographic groups. While there exist a bunch of methods that aim to reduce discrimination when abstaining, there is no mechanism that can do so in an explainable way. In this paper, we fill this gap by introducing Interpretable and Fair Abstaining Classifier IFAC, an algorithm that can reject predictions both based on their uncertainty and their unfairness. By rejecting possibly unfair predictions, our method reduces error and positive decision rate differences across demographic groups of the non-rejected data. Since the unfairness-based rejections are based on an interpretable-by-design method, i.e., rule-based fairness checks and situation testing, we create a transparent process that can empower human decision-makers to review the unfair predictions and make more just decisions for them. This explainable aspect is especially important in light of recent AI regulations, mandating that any high-risk decision task should be overseen by human experts to reduce discrimination risks.

Figures (8)

• Figure 1: Intuition behind IFAC
• Figure 2: The four steps for learning IFAC.
• Figure 3: Performance measures over demographic groups when applying a Random Forest in combination with various selective classifiers on ACSIncome (above) and WisconsinRecidivism (below). A regular UBAC increases differences in error- as well as positive decision rates among groups. Using IFAC, and rejecting instances based on unfairness, diminishes these differences.
• Figure 4: Examples for ACSIncome (left) and WisconsinRecidivism (right) of two rejected instances, and the explanation behind their rejections.
• Figure 5: Effects of $c$ and $w_u$ parameters in our selective classification settings.

Theorems & Definitions (1)

• proof