Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dynamic Feature Selection based on Rule-based Learning for Explainable Classification with Uncertainty Quantification

Javier Fumanal-Idocin, Raquel Fernandez-Peralta, Javier Andreu-Perez

TL;DR

This work addresses the need for explainable dynamic feature selection by formalizing uncertainty sources specific to sequential feature acquisition and adapting a rule-based global classifier for DFS. It introduces a model-agnostic DFS policy that greedily selects features to minimize divergence from a global rule-based predictor while accounting for epistemic costs, and uses an ensemble-/fuzzy-rule-based approach to quantify uncertainty during rule firings. Empirical results on five tabular datasets show competitive accuracy against state-of-the-art greedy and RL-based DFS methods, with the added benefit of transparent, interpretable predictions. The findings also reveal that predictive entropy decreases with more features largely due to information accumulation rather than the policy's effectiveness, and calibration behavior can diverge from confidence gains, highlighting important considerations for risk-sensitive applications like clinical decision support.

Abstract

Dynamic feature selection (DFS) offers a compelling alternative to traditional, static feature selection by adapting the selected features to each individual sample. This provides insights into the decision-making process for each case, which makes DFS especially significant in settings where decision transparency is key, i.e., clinical decisions. However, existing DFS methods use opaque models, which hinder their applicability in real-life scenarios. DFS also introduces new own sources of uncertainty compared to the static setting, which is also not considered in the existing literature. In this paper, we formalize the additional sources of uncertainty in DFS, and give formulas to estimate them. We also propose novel approach by leveraging a rule-based system as a base classifier for the DFS process, which enhances decision interpretability compared to neural estimators. Finally, we demonstrate the competitive performance of our rule-based DFS approach against established and state-of-the-art greedy and reinforcement learning methods, which are mostly considered opaque, compared to our explainable rulebased system.

Dynamic Feature Selection based on Rule-based Learning for Explainable Classification with Uncertainty Quantification

TL;DR

This work addresses the need for explainable dynamic feature selection by formalizing uncertainty sources specific to sequential feature acquisition and adapting a rule-based global classifier for DFS. It introduces a model-agnostic DFS policy that greedily selects features to minimize divergence from a global rule-based predictor while accounting for epistemic costs, and uses an ensemble-/fuzzy-rule-based approach to quantify uncertainty during rule firings. Empirical results on five tabular datasets show competitive accuracy against state-of-the-art greedy and RL-based DFS methods, with the added benefit of transparent, interpretable predictions. The findings also reveal that predictive entropy decreases with more features largely due to information accumulation rather than the policy's effectiveness, and calibration behavior can diverge from confidence gains, highlighting important considerations for risk-sensitive applications like clinical decision support.

Abstract

Dynamic feature selection (DFS) offers a compelling alternative to traditional, static feature selection by adapting the selected features to each individual sample. This provides insights into the decision-making process for each case, which makes DFS especially significant in settings where decision transparency is key, i.e., clinical decisions. However, existing DFS methods use opaque models, which hinder their applicability in real-life scenarios. DFS also introduces new own sources of uncertainty compared to the static setting, which is also not considered in the existing literature. In this paper, we formalize the additional sources of uncertainty in DFS, and give formulas to estimate them. We also propose novel approach by leveraging a rule-based system as a base classifier for the DFS process, which enhances decision interpretability compared to neural estimators. Finally, we demonstrate the competitive performance of our rule-based DFS approach against established and state-of-the-art greedy and reinforcement learning methods, which are mostly considered opaque, compared to our explainable rulebased system.

Paper Structure

This paper contains 22 sections, 3 theorems, 32 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Uncertainty Quantification in Dynamic Feature Selection
  3. Model Adaptation creates Epistemic Uncertainty
  4. Estimating Aleatoric Uncertainty during Sequential Acquisition
  5. Non-Monotonicity of Uncertainty in Feature Acquisition
  6. A Dynamic Feature Selection Procedure for Rule-Based Models
  7. Adapting a rule-based model for DFS
  8. Defining the DFS policy given a global model
  9. Difference Between Global and sub-Model Prediction
  10. Epistemic uncertainty concerning Rule Firings
  11. Estimating the Value Function in Test Time
  12. Experimentation
  13. Datasets and methods
  14. Results
  15. Conclusions and Future Lines
  16. ...and 7 more sections

Key Result

Lemma 1

where $\Delta u_i = u(\mathbf{x}_S \cup \{x_i\}) - u(\mathbf{x}_S)$.

Figures (5)

  • Figure 1: Performance comparison under varying feature costs and acquisition budgets for five tabular datasets.
  • Figure 2: Average predictive entropy for each model in all tabular datasets tested, for increasing percentages of the total budget.
  • Figure 3: Evolution of average ECE for increasing budgets in all tabular datasets tested.
  • Figure 4: Evolution of average accuracy difference between feature selected and random masks for increasing budgets in the tabular datasets studied.
  • Figure 5: Results for Dynamic Feature Selection using fuzzy rule-based classification (top row) and CART (bottom row). The blue line is accuracy, the green one is prediction uncertainty, and the red one is epistemic uncertainty.

Theorems & Definitions (5)

  • Lemma 1
  • proof
  • Proposition 1
  • proof
  • Corollary 1