Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond the Single-Best Model: Rashomon Partial Dependence Profile for Trustworthy Explanations in AutoML

Mustafa Cavus, Jan N. van Rijn, Przemysław Biecek

TL;DR

This work proposes a novel framework that incorporates model multiplicity into explanation generation by aggregating partial dependence profiles from a set of near optimal models, known as the Rashomon set, and suggests that Rashomon PDP improves the reliability and trustworthiness of model interpretations by adding additional information that would otherwise be neglected.

Abstract

Automated machine learning systems efficiently streamline model selection but often focus on a single best-performing model, overlooking explanation uncertainty, an essential concern in human centered explainable AI. To address this, we propose a novel framework that incorporates model multiplicity into explanation generation by aggregating partial dependence profiles (PDP) from a set of near optimal models, known as the Rashomon set. The resulting Rashomon PDP captures interpretive variability and highlights areas of disagreement, providing users with a richer, uncertainty aware view of feature effects. To evaluate its usefulness, we introduce two quantitative metrics, the coverage rate and the mean width of confidence intervals, to evaluate the consistency between the standard PDP and the proposed Rashomon PDP. Experiments on 35 regression datasets from the OpenML CTR23 benchmark suite show that in most cases, the Rashomon PDP covers less than 70% of the best model's PDP, underscoring the limitations of single model explanations. Our findings suggest that Rashomon PDP improves the reliability and trustworthiness of model interpretations by adding additional information that would otherwise be neglected. This is particularly useful in high stakes domains where transparency and confidence are critical.

Beyond the Single-Best Model: Rashomon Partial Dependence Profile for Trustworthy Explanations in AutoML

TL;DR

This work proposes a novel framework that incorporates model multiplicity into explanation generation by aggregating partial dependence profiles from a set of near optimal models, known as the Rashomon set, and suggests that Rashomon PDP improves the reliability and trustworthiness of model interpretations by adding additional information that would otherwise be neglected.

Abstract

Automated machine learning systems efficiently streamline model selection but often focus on a single best-performing model, overlooking explanation uncertainty, an essential concern in human centered explainable AI. To address this, we propose a novel framework that incorporates model multiplicity into explanation generation by aggregating partial dependence profiles (PDP) from a set of near optimal models, known as the Rashomon set. The resulting Rashomon PDP captures interpretive variability and highlights areas of disagreement, providing users with a richer, uncertainty aware view of feature effects. To evaluate its usefulness, we introduce two quantitative metrics, the coverage rate and the mean width of confidence intervals, to evaluate the consistency between the standard PDP and the proposed Rashomon PDP. Experiments on 35 regression datasets from the OpenML CTR23 benchmark suite show that in most cases, the Rashomon PDP covers less than 70% of the best model's PDP, underscoring the limitations of single model explanations. Our findings suggest that Rashomon PDP improves the reliability and trustworthiness of model interpretations by adding additional information that would otherwise be neglected. This is particularly useful in high stakes domains where transparency and confidence are critical.

Paper Structure

This paper contains 18 sections, 8 equations, 3 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methods
  3. Model Training and Rashomon Set Formation
  4. Rashomon Set Size
  5. Rashomon Ratio
  6. Partial Dependence Profile
  7. Rashomon Partial Dependence Profile
  8. Experimental Setup
  9. Datasets
  10. Setup
  11. Metrics
  12. Mean Width of Confidence Intervals
  13. Coverage Rate
  14. Results
  15. Highlighted Examples
  16. ...and 3 more sections

Figures (3)

  • Figure 1: Visual representation of the results across datasets. Each scatter represents a dataset and reveals the Rashomon ratio and coverage rate of that dataset. Most importantly, we have calculated the Spearman’s rank correlation between the Rashomon ratio and coverage rate across datasets.
  • Figure 2: Rashomon PDP on the variable surface_area of the dataset energy_efficiency.
  • Figure 3: Rashomon PDP on the variable temp of the dataset forest_fires.