Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sparse and Faithful Explanations Without Sparse Models

Yiyang Sun, Zhi Chen, Vittorio Orlandi, Tong Wang, Cynthia Rudin

TL;DR

This work introduces the Sparse Explanation Value (SEV), a new way of measuring sparsity in machine learning models that is defined using movements over a hypercube, allowing SEV to be defined consistently over various model classes, with movement restrictions reflecting real-world constraints.

Abstract

Even if a model is not globally sparse, it is possible for decisions made from that model to be accurately and faithfully described by a small number of features. For instance, an application for a large loan might be denied to someone because they have no credit history, which overwhelms any evidence towards their creditworthiness. In this work, we introduce the Sparse Explanation Value (SEV), a new way of measuring sparsity in machine learning models. In the loan denial example above, the SEV is 1 because only one factor is needed to explain why the loan was denied. SEV is a measure of decision sparsity rather than overall model sparsity, and we are able to show that many machine learning models -- even if they are not sparse -- actually have low decision sparsity, as measured by SEV. SEV is defined using movements over a hypercube, allowing SEV to be defined consistently over various model classes, with movement restrictions reflecting real-world constraints. We proposed the algorithms that reduce SEV without sacrificing accuracy, providing sparse and completely faithful explanations, even without globally sparse models.

Sparse and Faithful Explanations Without Sparse Models

TL;DR

This work introduces the Sparse Explanation Value (SEV), a new way of measuring sparsity in machine learning models that is defined using movements over a hypercube, allowing SEV to be defined consistently over various model classes, with movement restrictions reflecting real-world constraints.

Abstract

Even if a model is not globally sparse, it is possible for decisions made from that model to be accurately and faithfully described by a small number of features. For instance, an application for a large loan might be denied to someone because they have no credit history, which overwhelms any evidence towards their creditworthiness. In this work, we introduce the Sparse Explanation Value (SEV), a new way of measuring sparsity in machine learning models. In the loan denial example above, the SEV is 1 because only one factor is needed to explain why the loan was denied. SEV is a measure of decision sparsity rather than overall model sparsity, and we are able to show that many machine learning models -- even if they are not sparse -- actually have low decision sparsity, as measured by SEV. SEV is defined using movements over a hypercube, allowing SEV to be defined consistently over various model classes, with movement restrictions reflecting real-world constraints. We proposed the algorithms that reduce SEV without sacrificing accuracy, providing sparse and completely faithful explanations, even without globally sparse models.
Paper Structure (26 sections, 1 theorem, 11 equations, 54 figures, 29 tables)

This paper contains 26 sections, 1 theorem, 11 equations, 54 figures, 29 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Sparse Explanation Values
  4. Optimizing Sparse Explanation Values
  5. Volume-based SEV$^+$ Optimization for Linear Models
  6. Individual-based SEV Optimization
  7. Experiments
  8. Most models already have low SEVs.
  9. SEV can be reduced with no loss in performance.
  10. It is unnecessary to have global sparsity to have sparse explanations.
  11. Explaining a fixed model: comparing SEV to other local explanation methods
  12. Restricted SEV in practice
  13. Discussion and Limitations
  14. Datasets Description
  15. Model Training
  16. ...and 11 more sections

Key Result

Theorem 5

Consider a linear classifier, $f(\boldsymbol{x}):= \mathbf{1}[(\beta_0 + \sum_{j=1}^p \beta_j x_j)>0]$, where $\forall j$, $\beta_j\neq 0$, and for reference $\tilde{\boldsymbol{x}}$, we have $f(\tilde{\boldsymbol{x}}) = 0$ (i.e., reference predicted as negative). Let $g^{\textrm{\rm ref}}(\boldsymb where $c_k$ is a finite constant unrelated to the $\beta$'s.

Figures (54)

  • Figure 1: Visual illustrations of SEV$^{}$ definitions
  • Figure 2: SEV$^{+}$ performance for linear classifiers in Adult
  • Figure 3: Performance of All-Opt$^{+}$ and All-Opt$^{-}$ in Adult
  • Figure 4: Local Explanation Methods' Performance Comparison in COMPAS
  • Figure 5: The performance of All-Opt$^{\circledR}$ on COMPAS data
  • ...and 49 more figures

Theorems & Definitions (5)

  • Definition 1: SEV$^{}$ hypercube
  • Definition 2: SEV Plus, denoted SEV$^{+}$
  • Definition 3: SEV Minus, denoted SEV$^{-}$
  • Definition 4: Restricted SEV, denoted SEV$^{}$$^\circledR$
  • Theorem 5