Feature Relevancy, Necessity and Usefulness: Complexity and Algorithms

TL;DR

This work redefines feature importance within a logic-based explainability framework by formalizing relevancy, necessity, and a novel global usefulness through sufficient reasons $SR(M,e)$. It extends tractability results to decision trees with mixed feature types and proves linear-time algorithms for necessity in DTs and FBDDs, while establishing a quadratic-time method for relevancy in DTs and linking usefulness to model equivalence. A practical usefulness scoring method is proposed and validated on three public datasets, showing competitive alignment with ground-truth importance and SHAP rankings, while offering substantial computational efficiency. The results advance XAI by providing a principled, scalable approach to feature importance that integrates local explanations with a global usefulness perspective, and by highlighting remaining gaps for more expressive circuits like FBDDs and d-DNNFs.

Abstract

Given a classification model and a prediction for some input, there are heuristic strategies for ranking features according to their importance in regard to the prediction. One common approach to this task is rooted in propositional logic and the notion of \textit{sufficient reason}. Through this concept, the categories of relevant and necessary features were proposed in order to identify the crucial aspects of the input. This paper improves the existing techniques and algorithms for deciding which are the relevant and/or necessary features, showing in particular that necessity can be detected efficiently in complex models such as neural networks. We also generalize the notion of relevancy and study associated problems. Moreover, we present a new global notion (i.e. that intends to explain whether a feature is important for the behavior of the model in general, not depending on a particular input) of \textit{usefulness} and prove that it is related to relevancy and necessity. Furthermore, we develop efficient algorithms for detecting it in decision trees and other more complex models, and experiment on three datasets to analyze its practical utility.

Figures (7)

• Figure 1: Example of a decision tree representing a recommendation system for a film database. The set of features is Dur (duration), Rate, Year and Hst (whether the film is of the historical genre). The domains are $\mathcal{D}_{\texttt{Dur}} = [0, \infty]$, $\mathcal{D}_{\texttt{Rate}} = [0,1]$, $\mathcal{D}_{\texttt{Year}} = [1888, \infty]$ and $\mathcal{D}_{\texttt{Hst}} = \{0, 1\}$. Note that the first three features are numerical, while the last one is categorical. The tree classifies entity $e = \{\texttt{Dur}:90, \texttt{Rate}:0.85, \texttt{Year}:2005, \texttt{Hst}: 0\}$ to $1$. Moreover, the sufficient reasons for the result are $\{\texttt{Dur}, \texttt{Rate}\}$ and $\{\texttt{Dur}, \texttt{Year}\}$, and therefore $\texttt{Dur}$ is a necessary feature while $\texttt{Rate}$ and $\texttt{Year}$ are both relevant.
• Figure 2: An FBDD representing the CNF formula from Example \ref{['example:CNF']}. Note that each directed path does not contain two nodes with the same feature.
• Figure 3: $\texttt{taut}{}_T(v, e, x, (a, b])$
• Figure 4: $\texttt{taut}{}_T(v, e, x, Z)$
• Figure 5: Results for the California Housing Dataset. For each number of bins we train 20 models and show the average score of each feature across all of them, alongside the Q1 and Q3 quartiles. We also display the average accuracy of the models.

Theorems & Definitions (44)

• Example 1
• Definition 1: Sufficient Reason marques2023logic
• Example 2
• Definition 2: Relevant feature
• Definition 3: Necessary feature
• Definition 4: Useful feature
• Example 3
• Definition 5: Decision tree
• Definition 6: FBDD (Free Binary Decision Diagram)
• Definition 7: CNF of a boolean decision tree