Practical Attribution Guidance for Rashomon Sets

TL;DR

This work addresses the practical challenge of exploring Rashomon sets—collections of near-optimal models with potentially conflicting explanations—by formalizing two core axioms: generalizability and implementation sparsity. It introduces the General Rashomon Subset Sampling (GRSS) framework, which leverages an ε-subgradient optimization approach to sample a generalized Rashomon set represented via input- or output-activated transformations of the data, along with a model- and task-agnostic generalized feature-attribution mechanism. The authors propose metrics such as SER, dist_inf, and FER to quantify search efficiency and attribution diversity, and they validate the framework on synthetic and real-world datasets, showing that it can recover meaningful, non-redundant attributions across model families and epsilons. The approach offers a practical, theory-backed pathway to interpretable AI that accounts for model-interpretation variability, enabling more robust explanations and decision-making in complex tasks.

Abstract

Different prediction models might perform equally well (Rashomon set) in the same task, but offer conflicting interpretations and conclusions about the data. The Rashomon effect in the context of Explainable AI (XAI) has been recognized as a critical factor. Although the Rashomon set has been introduced and studied in various contexts, its practical application is at its infancy stage and lacks adequate guidance and evaluation. We study the problem of the Rashomon set sampling from a practical viewpoint and identify two fundamental axioms - generalizability and implementation sparsity that exploring methods ought to satisfy in practical usage. These two axioms are not satisfied by most known attribution methods, which we consider to be a fundamental weakness. We use the norms to guide the design of an $ε$-subgradient-based sampling method. We apply this method to a fundamental mathematical problem as a proof of concept and to a set of practical datasets to demonstrate its ability compared with existing sampling methods.

Figures (20)

• Figure 1: The pipeline at the top illustrates the overall Rashomon sampling process, highlighting the desired axioms at the corresponding positions. Detailed explanations are provided in the bottom dotted boxes. The first dotted box at the left shows the common XAI workflow, where researchers use a single ML model to derive explanations. The box at the top right outlines the conventional Rashomon set sampling workflow, where the structure of the reference model is fixed and assumed with prior knowledge, and then a model class is explored according to existing methods to approximate the Rashomon set. Here we use grayscale gradients to illustrate varying levels of interpretability among different ML structures. For example, a decision tree is generally interpretable, whereas an ensemble of trees tends to be less so. In practice, a pre-trained model is often provided as a black-box model, extended in the bottom right box, which could include NNs, decision trees, and other ML models identified for the same task. Therefore, a general workflow is proposed and tested in various experimental settings in Sec. \ref{['sec:experiments']}.
• Figure 2: An example of binary classification. Each object is characterized by features $x$1, $x$2, and shape. Possible models in the Rashomon set depend on different features for a simple circle and triangle classification.
• Figure 3: Illustration of model flexibility and explanation flexibility across different learning paradigms—supervised, unsupervised, and ensemble learning—alongside visualizations of first-order and second-order explanation spaces from our proposed framework. The $x$-axis represents the importance value of features, while the $y$-axis denotes corresponding features, with the point color indicating the magnitude of loss (darker shades denote higher loss).
• Figure 4: Summary of the Rashomon subset from a set of epsilons on the dataset COMPAS, including (a) the number of sampled models on different methods, where no bar indicates too many models (greater than 20,000 models) (b) the first-order FER, where the $x$-axis is epsilon (benchmarking against optimal loss) and colors represent according to SER, (c) the second-order FER, following the same format as above, (d) the detailed first-order feature attribution on individual features when epsilon is set as 0.05, where the vertical bars represent the bounds and the dotted lines connect the average scores, and (e) the detailed second-order feature attribution on feature pairs when epsilon is set 0.05, where the dotted lines represent the bounds and the solid lines connect the average scores. Each color corresponds to a sampling method and due to space limitations, some interaction pairs are omitted on the $x$-axis.
• Figure 5: The detailed second-order feature attribution on interested feature pairs.

Theorems & Definitions (25)

• Remark 1
• Definition 2: generalized Rashomon set
• Theorem 3
• Remark 4
• Proposition 5
• Definition 6: generalized feature attribution
• Definition 7: Feature attribution set and feature attribution space
• Theorem 8
• Remark 9
• Definition 10: Searching efficiency ratio