Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graphical Perception of Saliency-based Model Explanations

Yayan Zhao, Mingwei Li, Matthew Berger

TL;DR

The findings show that factors related to visualization design decisions, the type of alignment, and qualities of the saliency map all play important roles in how humans perceive saliency-based visual explanations.

Abstract

In recent years, considerable work has been devoted to explaining predictive, deep learning-based models, and in turn how to evaluate explanations. An important class of evaluation methods are ones that are human-centered, which typically require the communication of explanations through visualizations. And while visualization plays a critical role in perceiving and understanding model explanations, how visualization design impacts human perception of explanations remains poorly understood. In this work, we study the graphical perception of model explanations, specifically, saliency-based explanations for visual recognition models. We propose an experimental design to investigate how human perception is influenced by visualization design, wherein we study the task of alignment assessment, or whether a saliency map aligns with an object in an image. Our findings show that factors related to visualization design decisions, the type of alignment, and qualities of the saliency map all play important roles in how humans perceive saliency-based visual explanations.

Graphical Perception of Saliency-based Model Explanations

TL;DR

The findings show that factors related to visualization design decisions, the type of alignment, and qualities of the saliency map all play important roles in how humans perceive saliency-based visual explanations.

Abstract

In recent years, considerable work has been devoted to explaining predictive, deep learning-based models, and in turn how to evaluate explanations. An important class of evaluation methods are ones that are human-centered, which typically require the communication of explanations through visualizations. And while visualization plays a critical role in perceiving and understanding model explanations, how visualization design impacts human perception of explanations remains poorly understood. In this work, we study the graphical perception of model explanations, specifically, saliency-based explanations for visual recognition models. We propose an experimental design to investigate how human perception is influenced by visualization design, wherein we study the task of alignment assessment, or whether a saliency map aligns with an object in an image. Our findings show that factors related to visualization design decisions, the type of alignment, and qualities of the saliency map all play important roles in how humans perceive saliency-based visual explanations.
Paper Structure (32 sections, 2 equations, 11 figures)

This paper contains 32 sections, 2 equations, 11 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Model explanations
  4. Evaluation of visual explanations
  5. Perceptual factors in visualizing quantitative data
  6. Perception of Alignment: Task and Hypotheses
  7. Task
  8. Hypotheses
  9. Experimental Design: Methods
  10. Visually encoding saliency maps
  11. Computing and categorizing alignment
  12. Characterizing saliency maps
  13. Measuring shape complexity
  14. User Study
  15. Preparation
  16. ...and 17 more sections

Figures (11)

  • Figure 1: We place the focus of our work (highlighted in blue) within the broader context of human-AI collaboration. Given a particular task, e.g. decide on whether a model will predict a given category, (I.) a human will first perceive the image for task-relevant features on the input, and perceive the graphical encoding of the visual explanation for features important to the model. Next (II.) a human will assess alignment between image features, and model-derived features. From here (III.), a human will then make a decision on the provided task. Studying the perception of alignment provides us with a better understanding of human decisions made in AI-assisted tasks, whether a human's decision is consistent, or inconsistent, with a model's prediction.
  • Figure 2: In cases where saliency-based model explanations align well with the object (top row) or not (middle row), we do not anticipate visualization playing a role in perception. However, when model explanations are imperfect, giving rise to ambiguity in alignment (bottom row), we anticipate that how saliency maps are visually encoded will impact human perception.
  • Figure 3: We study visualization designs along two axes: different rows indicate the amount of detail from a saliency map that a visualization can display, here shown in increasing amount through binary masks, contour-based visualizations, and heatmap visualizations. Different columns convey the amount of detail specified for the data domain in a visualization.
  • Figure 4: a) Alignment type: We categorize alignment into three types: (i) overestimation; (ii) underestimation and (iii) partial alignment. b) Shape simplicity: We measure shape complexity by ratio of two perimeters. Left to right: simple shapes to complex shapes according to the shape simplicity measure. c) Entropy: We characterize saliency maps by the spatial distribution of saliency values through entropy. Left to right: low entropy to high entropy, shown as heatmaps.
  • Figure 5: User interface in the study.
  • ...and 6 more figures