A Guide to Feature Importance Methods for Scientific Inference

TL;DR

This paper addresses the challenge of interpreting feature importance (FI) methods for scientific inference, highlighting that different FI approaches correspond to different feature–target associations and can mislead if misapplied. It offers a comprehensive, loss-based, model-agnostic guide that maps FI methods to specific association questions, provides formal interpretation rules, and delivers proofs, simulations, and practical recommendations. The core contributions classify FI into univariate perturbations (PFI/CFI), marginalization-based (SAGE), and refitting-based (LOCO/WVIM) families, detailing when each yields insights about unconditional or conditional dependencies and how to quantify uncertainty. By clarifying these mappings and outlining uncertainty estimation approaches, the work enables researchers to select FI methods appropriate for rigorous DGP inference and points toward future directions for statistically principled inference from black-box predictors.

Abstract

While machine learning (ML) models are increasingly used due to their high predictive power, their use in understanding the data-generating process (DGP) is limited. Understanding the DGP requires insights into feature-target associations, which many ML models cannot directly provide due to their opaque internal mechanisms. Feature importance (FI) methods provide useful insights into the DGP under certain conditions. Since the results of different FI methods have different interpretations, selecting the correct FI method for a concrete use case is crucial and still requires expert knowledge. This paper serves as a comprehensive guide to help understand the different interpretations of global FI methods. Through an extensive review of FI methods and providing new proofs regarding their interpretation, we facilitate a thorough understanding of these methods and formulate concrete recommendations for scientific inference. We conclude by discussing options for FI uncertainty estimation and point to directions for future research aiming at full statistical inference from black-box ML models.

Figures (1)

• Figure 1: Six most important features following (a) PFI and (b) LOCO.