Enhancing Variable Importance in Random Forests: A Novel Application of Global Sensitivity Analysis
Giulia Vannucci, Roberta Siciliano, Andrea Saltelli
TL;DR
The paper tackles interpretability in Random Forests by aligning variable importance with the data-generating process via Global Sensitivity Analysis. It proposes RF_GS-VI, a total-sensitivity-index-based variable ranking that uses Saltelli-style sampling and RMSE as the fitness measure to quantify the influence of each predictor. Across three simulated scenarios and two real datasets, RF_GS-VI generally yields correct or more informative rankings than standard VI measures, while highlighting variables that traditional VI may overlook. The work advances Explainable AI by providing a principled, generative view of feature influence and suggests avenues for broader application and theory development.
Abstract
The present work provides an application of Global Sensitivity Analysis to supervised machine learning methods such as Random Forests. These methods act as black boxes, selecting features in high--dimensional data sets as to provide accurate classifiers in terms of prediction when new data are fed into the system. In supervised machine learning, predictors are generally ranked by importance based on their contribution to the final prediction. Global Sensitivity Analysis is primarily used in mathematical modelling to investigate the effect of the uncertainties of the input variables on the output. We apply it here as a novel way to rank the input features by their importance to the explainability of the data generating process, shedding light on how the response is determined by the dependence structure of its predictors. A simulation study shows that our proposal can be used to explore what advances can be achieved either in terms of efficiency, explanatory ability, or simply by way of confirming existing results.