Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dissenting Explanations: Leveraging Disagreement to Reduce Model Overreliance

Omer Reingold, Judy Hanwen Shen, Aditi Talati

TL;DR

This work addresses the fragility of explanations for black-box models by introducing dissenting explanations—explanations for opposing predictions from disagreeing models. It demonstrates, via a human study on deceptive hotel reviews, that providing dissenting explanations reduces human overreliance without sacrificing accuracy and can even lower trust in incorrect AI suggestions. The authors then present model-agnostic global and input-specific local methods to induce predictive and explanatory disagreement, showing that increased disagreement correlates with decreased overlap in explanations. The findings highlight a path toward more calibrated human-AI collaboration by treating explanations as arguments for and against a model’s decision, while outlining practical limitations and avenues for future work across tasks and domains.

Abstract

While explainability is a desirable characteristic of increasingly complex black-box models, modern explanation methods have been shown to be inconsistent and contradictory. The semantics of explanations is not always fully understood - to what extent do explanations "explain" a decision and to what extent do they merely advocate for a decision? Can we help humans gain insights from explanations accompanying correct predictions and not over-rely on incorrect predictions advocated for by explanations? With this perspective in mind, we introduce the notion of dissenting explanations: conflicting predictions with accompanying explanations. We first explore the advantage of dissenting explanations in the setting of model multiplicity, where multiple models with similar performance may have different predictions. In such cases, providing dissenting explanations could be done by invoking the explanations of disagreeing models. Through a pilot study, we demonstrate that dissenting explanations reduce overreliance on model predictions, without reducing overall accuracy. Motivated by the utility of dissenting explanations we present both global and local methods for their generation.

Dissenting Explanations: Leveraging Disagreement to Reduce Model Overreliance

TL;DR

This work addresses the fragility of explanations for black-box models by introducing dissenting explanations—explanations for opposing predictions from disagreeing models. It demonstrates, via a human study on deceptive hotel reviews, that providing dissenting explanations reduces human overreliance without sacrificing accuracy and can even lower trust in incorrect AI suggestions. The authors then present model-agnostic global and input-specific local methods to induce predictive and explanatory disagreement, showing that increased disagreement correlates with decreased overlap in explanations. The findings highlight a path toward more calibrated human-AI collaboration by treating explanations as arguments for and against a model’s decision, while outlining practical limitations and avenues for future work across tasks and domains.

Abstract

While explainability is a desirable characteristic of increasingly complex black-box models, modern explanation methods have been shown to be inconsistent and contradictory. The semantics of explanations is not always fully understood - to what extent do explanations "explain" a decision and to what extent do they merely advocate for a decision? Can we help humans gain insights from explanations accompanying correct predictions and not over-rely on incorrect predictions advocated for by explanations? With this perspective in mind, we introduce the notion of dissenting explanations: conflicting predictions with accompanying explanations. We first explore the advantage of dissenting explanations in the setting of model multiplicity, where multiple models with similar performance may have different predictions. In such cases, providing dissenting explanations could be done by invoking the explanations of disagreeing models. Through a pilot study, we demonstrate that dissenting explanations reduce overreliance on model predictions, without reducing overall accuracy. Motivated by the utility of dissenting explanations we present both global and local methods for their generation.
Paper Structure (38 sections, 6 equations, 7 figures, 5 tables)

This paper contains 38 sections, 6 equations, 7 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. One Model, Multiple Explanations
  4. Similar Models, Conflicting Explanations
  5. Overreliance and Human-AI Collaboration
  6. Model and Framework
  7. Motivating Study: The Importance of Dissenting Explanations
  8. Hypothesis
  9. Study Design
  10. Task Selection
  11. Deceptive Reviews Task
  12. Generating Explanations
  13. Conditions
  14. Participants
  15. Results
  16. ...and 23 more sections

Figures (7)

  • Figure 1: The process of generating explanations for the 4 conditions. The review text is for conceptual understanding only.
  • Figure 2: (a) Accuracy, (b) Overreliance, and (c) Agreement (Cohen's $\kappa$ score) for each experimental condition demonstrating that dissenting explanations $C_2$ significantly reduce overreliance without reducing overall accuracy. Error bars represent $95\%$ confidence intervals of the mean across participants (N=178).
  • Figure 3: Level of trust reported by participants on a scale of (1) "not at all" to (5) "a great deal". The level of trust was significantly lower for $C_2$ (dissenting explanations condition)
  • Figure 4: As we emphasize the importance of model predictive disagreement through increasing $\lambda$, the agreement between explanations as measured by the overlap in top features also decreases.
  • Figure 5: As we emphasize the importance of model predictive disagreement through increasing $\lambda$, the agreement between explanations as measured by the overlap in top features also decreases.
  • ...and 2 more figures

Theorems & Definitions (4)

  • Definition 1: Dissenting Explanation
  • Definition 2: Global Predictive Disagreement
  • Remark 3
  • Remark 5