Unlocking the Black Box: A Five-Dimensional Framework for Evaluating Explainable AI in Credit Risk

TL;DR

This work addresses the tension between predictive power and regulatory explainability in credit risk by applying SHAP and LIME to three models (LR, RF, NN) on the Prosper dataset, augmented with macroeconomic factors. It introduces a novel five-dimensional framework—Inherent Interpretability, Global Explanations, Local Explanations, Consistency, and Complexity—to evaluate explainability beyond accuracy. Empirically, the Neural Network delivers the strongest default prediction performance, while post-hoc explanations enable compliant, instance-level insights crucial for regulatory and business stakeholders. The study demonstrates the feasibility of deploying sophisticated models in regulated finance by pairing them with robust XAI methods and offers a scalable framework for communicating model behavior to diverse audiences. Future work includes broader dataset validation, direct comparisons of explainability techniques, and exploration of transformer-based architectures to extend applicability.

Abstract

The financial industry faces a significant challenge modeling and risk portfolios: balancing the predictability of advanced machine learning models, neural network models, and explainability required by regulatory entities (such as Office of the Comptroller of the Currency, Consumer Financial Protection Bureau). This paper intends to fill the gap in the application between these "black box" models and explainability frameworks, such as LIME and SHAP. Authors elaborate on the application of these frameworks on different models and demonstrates the more complex models with better prediction powers could be applied and reach the same level of the explainability, using SHAP and LIME. Beyond the comparison and discussion of performances, this paper proposes a novel five dimensional framework evaluating Inherent Interpretability, Global Explanations, Local Explanations, Consistency, and Complexity to offer a nuanced method for assessing and comparing model explainability beyond simple accuracy metrics. This research demonstrates the feasibility of employing sophisticated, high performing ML models in regulated financial environments by utilizing modern explainability techniques and provides a structured approach to evaluate the crucial trade offs between model performance and interpretability.

Figures (6)

• Figure 1: Distribution of Key Categorical Features.
• Figure 2: Treatment of Continuous Data (Before)
• Figure 3: Treatment of Continuous Data (After)
• Figure 4: Comparison of Performances
• Figure 5: Instance-wise explanation of Neural Network Model