What Makes a Good Explanation?: A Harmonized View of Properties of Explanations
Zixi Chen, Varshini Subhash, Marton Havasi, Weiwei Pan, Finale Doshi-Velez
TL;DR
The paper addresses the lack of standardization in explanation properties for interpretable ML by harmonizing and systematizing mathematical formulations across the literature. It identifies four core computational properties—robustness/sensitivity, faithfulness/fidelity, complexity/compactness, and homogeneity—and provides a framework to map and compare varying definitions within these categories, including function-based, feature-attribution, and example-based explanations. The authors survey existing formulations, normalize notation, and discuss trade-offs and tensions between properties (e.g., faithfulness vs. complexity, robustness vs. fidelity, and fairness implications in homogeneity). Their contributions yield a reference for practitioners and researchers to select task-appropriate formulations and to standardize reporting, enabling clearer comparisons and progress in interpretable ML. The work has practical impact by guiding method selection, evaluation, and future research toward coherent, context-aware explanation properties that can be rigorously assessed.
Abstract
Interpretability provides a means for humans to verify aspects of machine learning (ML) models and empower human+ML teaming in situations where the task cannot be fully automated. Different contexts require explanations with different properties. For example, the kind of explanation required to determine if an early cardiac arrest warning system is ready to be integrated into a care setting is very different from the type of explanation required for a loan applicant to help determine the actions they might need to take to make their application successful. Unfortunately, there is a lack of standardization when it comes to properties of explanations: different papers may use the same term to mean different quantities, and different terms to mean the same quantity. This lack of a standardized terminology and categorization of the properties of ML explanations prevents us from both rigorously comparing interpretable machine learning methods and identifying what properties are needed in what contexts. In this work, we survey properties defined in interpretable machine learning papers, synthesize them based on what they actually measure, and describe the trade-offs between different formulations of these properties. In doing so, we enable more informed selection of task-appropriate formulations of explanation properties as well as standardization for future work in interpretable machine learning.