Pushing the Boundaries of Interpretability: Incremental Enhancements to the Explainable Boosting Machine

TL;DR

The paper addresses the black-box problem in high-stakes domains by leveraging Explainable Boosting Machines (EBMs), a glassbox model that combines accuracy with transparency. It introduces three enhancements—targeted Bayesian hyperparameter optimization, a fairness-aware objective based on demographic parity, and a self-supervised pretraining pipeline using init_scores—to improve performance, fairness, and robustness on tabular datasets. Across Adult Income, Credit Card Fraud Detection, and UCI Heart Disease, the approach yields modest but reliable gains in ROC AUC, faster convergence, and reduced sensitivity to protected attributes, with pretraining notably stabilizing training and improving discrimination in low-label scenarios. The work provides a practical, interpretable workflow for trustworthy tabular modeling and points to future work on broader fairness definitions and cross-dataset validation.

Abstract

The widespread adoption of complex machine learning models in high-stakes domains has brought the "black-box" problem to the forefront of responsible AI research. This paper aims at addressing this issue by improving the Explainable Boosting Machine (EBM), a state-of-the-art glassbox model that delivers both high accuracy and complete transparency. The paper outlines three distinct enhancement methodologies: targeted hyperparameter optimization with Bayesian methods, the implementation of a custom multi-objective function for fairness for hyperparameter optimization, and a novel self-supervised pre-training pipeline for cold-start scenarios. All three methodologies are evaluated across standard benchmark datasets, including the Adult Income, Credit Card Fraud Detection, and UCI Heart Disease datasets. The analysis indicates that while the tuning process yielded marginal improvements in the primary ROC AUC metric, it led to a subtle but important shift in the model's decision-making behavior, demonstrating the value of a multi-faceted evaluation beyond a single performance score. This work is positioned as a critical step toward developing machine learning systems that are not only accurate but also robust, equitable, and transparent, meeting the growing demands of regulatory and ethical compliance.

Figures (3)

• Figure 1: Deriving init_scores.
• Figure 2: Before and After Fairness-Aware HPO
• Figure 3: Final Confusion Matrices for Adult, Credit card and Heart datasets respectively.