GRANITE: A Generalized Regional Framework for Identifying Agreement in Feature-Based Explanations

TL;DR

GRANITE addresses inconsistent feature-based explanations by partitioning the input space into regions where interaction and distribution influences are minimized, enabling consistent, interpretable regional explanations. It builds on a unifying fANOVA/Möbius framework with masking, behavior, and interaction operators across four design dimensions, and extends to feature groups. The method uses recursive partitioning to identify regions that minimize regional disagreement between explanations (e.g., full vs pure, conditional vs marginal), with estimations derived from empirical distributions and a tractable tree-based search. Experiments on real datasets show substantial reduction in disagreement between different explanation paradigms, improving interpretability while preserving faithfulness to data distribution.

Abstract

Feature-based explanation methods aim to quantify how features influence the model's behavior, either locally or globally, but different methods often disagree, producing conflicting explanations. This disagreement arises primarily from two sources: how feature interactions are handled and how feature dependencies are incorporated. We propose GRANITE, a generalized regional explanation framework that partitions the feature space into regions where interaction and distribution influences are minimized. This approach aligns different explanation methods, yielding more consistent and interpretable explanations. GRANITE unifies existing regional approaches, extends them to feature groups, and introduces a recursive partitioning algorithm to estimate such regions. We demonstrate its effectiveness on real-world datasets, providing a practical tool for consistent and interpretable feature explanations.

Figures (18)

• Figure 1: Disagreement of average absolute marginal and conditional SHAP lundberg_2017_unified and PredDiff Robnik2008 values for synthetic data, where $X_1$ interacts with $X_2$ and $X_4$ depends on $X_3$, cf. Section \ref{['sec:disagreement']}. Explanations disagree globally (bottom left) due to feature interactions and dependencies. Regional explanations remove disagreements caused by interactions (bottom right) and dependencies (top left), with overall agreement when combined (top right).
• Figure 2: Main effect of $x_1$ (left) and its interaction with $x_2$ (right) in region $\Omega_r$, where $F_{\mid\Omega_r}(x) = x_1 + 0.7\,x_1x_2$.
• Figure 3: Minimizing interaction disagreement with $\mathcal{B}_{loc,x_0}$ and $\mathcal{M}_m$ for: (a) Individual feature influence with full (ICE curves as thin lines) versus pure (PDP as thick line) shown globally (left) and regionally (right) for hour and temperature. (b) Interaction feature influence of hour and temperature for regions depending on workday.
• Figure 4: Minimizing interaction disagreement with $\mathcal{B}_{\text{risk}}$ and $\mathcal{M}_m$: GRANITE minimizes full ($\mathcal{I}_{\text{full}})$ versus pure ($\mathcal{I}_{\text{pure}}$) global risk, while the plots show first- and second-order partial risk (SAGE-like interactions) globally and regionally.
• Figure 5: Minimizing interaction disagreement with $\mathcal{B}_{sens}$ and $\mathcal{M}_m$: Total Sobol' ($\mathcal{I}_{\text{full}}$, upper bar) and closed Sobol' ($\mathcal{I}_{\text{pure}}$, lower bar) shown globally (left) and regionally (right) for top-$6$ features. Stacked bars sorted by size for the regions create a layered view of regional rankings.

Theorems & Definitions (13)

• Definition 1: Masking
• Definition 2: Behavior
• Definition 3: Interaction
• Definition 4: Disagreement Problem
• Definition 5: Regional Disagreement
• Theorem 1
• Theorem 2
• Theorem 3
• Corollary 1
• proof