Exploring the cloud of feature interaction scores in a Rashomon set

TL;DR

The paper tackles the problem that feature interactions explained from a single high-performing model may be unreliable. It introduces the feature interaction score (FIS) and its cloud (FISC) to quantify interactions across a Rashomon set of similarly accurate models, and develops a mask-based framework to explore these interactions. A greedy sampling algorithm plus Halo and swarm visualizations are proposed to characterize the boundary and variation of FISC, with demonstrations on synthetic data, recidivism prediction, and image classification. The results show substantial variation in interaction strength across models with comparable performance, offering a new lens for interpreting, comparing, and selecting models based on interaction structure and reliability.

Abstract

Interactions among features are central to understanding the behavior of machine learning models. Recent research has made significant strides in detecting and quantifying feature interactions in single predictive models. However, we argue that the feature interactions extracted from a single pre-specified model may not be trustworthy since: a well-trained predictive model may not preserve the true feature interactions and there exist multiple well-performing predictive models that differ in feature interaction strengths. Thus, we recommend exploring feature interaction strengths in a model class of approximately equally accurate predictive models. In this work, we introduce the feature interaction score (FIS) in the context of a Rashomon set, representing a collection of models that achieve similar accuracy on a given task. We propose a general and practical algorithm to calculate the FIS in the model class. We demonstrate the properties of the FIS via synthetic data and draw connections to other areas of statistics. Additionally, we introduce a Halo plot for visualizing the feature interaction variance in high-dimensional space and a swarm plot for analyzing FIS in a Rashomon set. Experiments with recidivism prediction and image classification illustrate how feature interactions can vary dramatically in importance for similarly accurate predictive models. Our results suggest that the proposed FIS can provide valuable insights into the nature of feature interactions in machine learning models.

Figures (17)

• Figure 1: Two accurate models with different feature interactions and an illustration of FIS in a hypothetical Rashomon set $\mathcal{R}(\epsilon, f^{*}, \mathcal{F})$.
• Figure 2: Exploring feature interaction of function $f = x_i + x_j + x_i * x_j$ and $f = x_i + x_j + x_k + x_i * x_j * x_k$ in Rashomon set. The data points are sampled from both functions. Exploring $\phi_{i}$ and $\phi_{j}$ separately enables us to draw red circles representing $\sum_{i \in I}\phi_{i} = \epsilon$, where $\epsilon$ is the radius, e.g. $\epsilon = 0.1$ can be potentially expressed by $\phi_{i} = 0.01$ and $\phi_{j} = 0.09$; a detailed example is provided in Appendix \ref{['sec:illustration_of_halo']}. From inner to outer, the radii are 0.1, 0.2, and 0.3, respectively and the corresponding joint effects $\tilde{\epsilon}$ can be calculated by $\phi_{i, j}$ and $\phi_{i, j, k}$, from which we plot the blue curves. This can be achieved in 2D (left) and higher dimensions, e.g. 3D (right).
• Figure 3: Pairwise Halo plot from FISC applied to an MLP trained from NID setting. The ground truth interaction defined in function $f(x) = \bigwedge (x; \{x_0^*, x_1^* \} \cup x'_2 ) + \bigwedge (x; \{x_i^{*}\}_{i=11}^{30}) +\sum_{j=1}^{40}x_j$ from left to right are: true, true, false, true, respectively.
• Figure 4: Pairwise Halo: 'prior' and 'juvenile' (top); and triplewise Halo: 'prior', 'juvenile' and 'charge' (bottom).
• Figure 5: Illustration of FISC using swarm plots, where the black vertical line is the threshold and the yellow star is the reference FIS. Each point in the plot is coloured based on loss value.

Theorems & Definitions (1)

• remark 1