Class-specific feature selection for classification explainability

TL;DR

A comprehensive review of the concept of class-specific, with a focus on feature selection and classification, and a novel class-specific relevance matrix, from which some more sophisticated classification schemes can be derived.

Abstract

Feature Selection techniques aim at finding a relevant subset of features that perform equally or better than the original set of features at explaining the behavior of data. Typically, features are extracted from feature ranking or subset selection techniques, and the performance is measured by classification or regression tasks. However, while selected features may not have equal importance for the task, they do have equal importance for each class. This work first introduces a comprehensive review of the concept of class-specific, with a focus on feature selection and classification. The fundamental idea of the class-specific concept resides in the understanding that the significance of each feature can vary from one class to another. This contrasts with the traditional class-independent approach, which evaluates the importance of attributes collectively for all classes. For example, in tumor prediction scenarios, each type of tumor may be associated with a distinct subset of relevant features. These features possess significant discriminatory power, enabling the differentiation of one tumor type from others. This class-specific perspective offers a more effective approach to classification tasks by recognizing and leveraging the unique characteristics of each class. Secondly, classification schemes from one-versus-all and one-versus-each strategies are described, and a novel deep one-versus-each strategy is introduced, which offers advantages from the point of view of explainability (feature selection) and decomposability (classification). Thirdly, a novel class-specific relevance matrix is presented, from which some more sophisticated classification schemes can be derived, such as the three-layer class-specific scheme. The potential for further advancements is wide and will open new horizons for exploring novel research directions in multiclass hyperdimensional contexts.

Figures (5)

• Figure 1: The taxonomy of approaches based on the number of feature subsets can be categorized into two primary types: 1 (the same set of features is employed universally across all L classes) or L (one for each L classes).
• Figure 2: Classification scheme for a traditional feature selection method, from the results in Table \ref{['tab:ex_ranking_traditional']}.
• Figure 3: Classification scheme from the results shown in Table \ref{['tab:ex_ranking']}.
• Figure 4: Two--layer scheme classification scheme from deep one--versus--each strategy.
• Figure 5: Three--layer class--specific classification scheme. (Red dashed lines refer to outputs from empty set of features.)