Optimal probabilistic feature shifts for reclassification in tree ensembles
Víctor Blanco, Alberto Japón, Justo Puerto, Peter Zhang
TL;DR
The paper addresses reclassifying a specific observation in tree ensemble classifiers by perturbing features under a limited effort budget. It moves beyond distance-based perturbation methods by introducing a probabilistic feasibility framework that allocates effort across features and models leaf-visit probabilities to maximize the chance of reaching a target class $k^*$. The authors develop several MINLP formulations, including most-likely-path and robust CVaR-/k-sum-inspired variants, to derive optimal feature shifts and to rank feature importance under feasibility constraints. A real-world obesity classification case study demonstrates that the proposed approaches yield meaningful, robust rankings and improved reclassification rates, with potential implications for intervention design and policy planning.
Abstract
In this paper we provide a novel mathematical optimization based methodology to perturb the features of a given observation to be re-classified, by a tree ensemble classification rule, to a certain desired class. The method is based on these facts: the most viable changes for an observation to reach the desired class do not always coincide with the closest distance point (in the feature space) of the target class; individuals put effort on a few number of features to reach the desired class; and each individual is endowed with a probability to change each of its features to a given value, which determines the overall probability of changing to the target class. Putting all together, we provide different methods to find the features where the individuals must exert effort to maximize the probability to reach the target class. Our method also allows us to rank the most important features in the tree-ensemble. The proposed methodology is tested on a real dataset, validating the proposal.