I-trustworthy Models. A framework for trustworthiness evaluation of probabilistic classifiers

TL;DR

The paper addresses reliability beyond accuracy for probabilistic classifiers used in inference tasks by formalizing $\mathcal{I}$-trustworthiness as local calibration on a task-relevant feature space $\mathcal{X}$. It introduces KLCE, a kernelized local calibration error, with an unbiased $\widehat{\mathrm{KLCE}^2}$ estimator and a bootstrap-based p-value for testing $\mathrm{KLCE}=0$, along with a convergence bound. Additionally, it proposes a diagnostic tool via Local Calibration Bias (LCB) to localize biases in $\mathcal{X}$ and demonstrates that conventional recalibration and multicalibration methods often fail to achieve local calibration in real data (e.g., COMPAS, American Housing Survey). The framework provides a principled approach to auditing and improving trustworthiness of probabilistic classifiers in inference tasks, with implications for fairness and regulatory oversight.

Abstract

As probabilistic models continue to permeate various facets of our society and contribute to scientific advancements, it becomes a necessity to go beyond traditional metrics such as predictive accuracy and error rates and assess their trustworthiness. Grounded in the competence-based theory of trust, this work formalizes I-trustworthy framework -- a novel framework for assessing the trustworthiness of probabilistic classifiers for inference tasks by linking local calibration to trustworthiness. To assess I-trustworthiness, we use the local calibration error (LCE) and develop a method of hypothesis-testing. This method utilizes a kernel-based test statistic, Kernel Local Calibration Error (KLCE), to test local calibration of a probabilistic classifier. This study provides theoretical guarantees by offering convergence bounds for an unbiased estimator of KLCE. Additionally, we present a diagnostic tool designed to identify and measure biases in cases of miscalibration. The effectiveness of the proposed test statistic is demonstrated through its application to both simulated and real-world datasets. Finally, LCE of related recalibration methods is studied, and we provide evidence of insufficiency of existing methods to achieve I-trustworthiness.

Figures (6)

• Figure 1: Top Panels. Consider the task of inferring the average age of homeowners when homeownership status is unknown (test sample), using two models that assign probabilities of homeownership. While Model 1 (blue model) outperforms Model 2 in terms of accuracy, Brier score, and ECE, only Model 2 provides an unbiased estimate of the inference target (right panel). The histograms are based on 200 realizations of test/train samples.
• Figure 2: Motivation 2. Consider the task of inferring the homeownership gap between Black and non-Black householders with two probabilistic models. Their performance is shown in the three left panels. Both models are untrustworthy (right panel), and this lack of trustworthiness cannot be hypothesis tested using Accuracy, Brier score, or ECE. The histograms are based on 200 realizations of test/train samples.
• Figure 3: Type-II Error as a function of dimension of $X$ and sample size $N$.
• Figure 4: Recidivism risk prediction results. Calibration error measures are shown for before calibration (left column) and after calibration. Reliability diagram (top panels) visualizes how well probabilities are calibrated. BS, ECE and MCE quantify how well a model is calibrated. ${\rm KLCE}$ test statistic can measure local calibration error. The corresponding $p$-value quantifies the statistical significance of ${\rm KLCE}^{2} = 0$.
• Figure 5: Type-I Error as a function of kernel width and sample size.

Theorems & Definitions (26)

• Definition 2.1: Calibration
• Definition 2.2: Local Calibration
• Definition 2.3: ${\mathcal{I}}$-trustworthy
• Definition 3.1
• Lemma 3.2
• Definition 3.3
• Theorem 3.4: ${\mathcal{I}}$-trustworthy Competency Theorem
• Corollary 3.5
• Theorem 3.6: Convergence Bound
• Corollary 3.7: ${\mathcal{I}}$-trustworthy Confidence Gaurantee