Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Understanding the Effect of Algorithm Transparency of Model Explanations in Text-to-SQL Semantic Parsing

Daking Rai, Rydia R. Weiland, Kayla Margaret Gabriella Herrera, Tyler H. Shaw, Ziyu Yao

TL;DR

Investigating explanations for a structured prediction task called ``text-to-SQL Semantic Parsing'', which translates a natural language question into a structured query language (SQL) program, shows that the low-/high-transparency explanations often lead to less/more user reliance on the model decisions, whereas the medium-transparency explanations strike a good balance.

Abstract

Explaining the decisions of AI has become vital for fostering appropriate user trust in these systems. This paper investigates explanations for a structured prediction task called ``text-to-SQL Semantic Parsing'', which translates a natural language question into a structured query language (SQL) program. In this task setting, we designed three levels of model explanation, each exposing a different amount of the model's decision-making details (called ``algorithm transparency''), and investigated how different model explanations could potentially yield different impacts on the user experience. Our study with $\sim$100 participants shows that (1) the low-/high-transparency explanations often lead to less/more user reliance on the model decisions, whereas the medium-transparency explanations strike a good balance. We also show that (2) only the medium-transparency participant group was able to engage further in the interaction and exhibit increasing performance over time, and that (3) they showed the least changes in trust before and after the study.

Understanding the Effect of Algorithm Transparency of Model Explanations in Text-to-SQL Semantic Parsing

TL;DR

Investigating explanations for a structured prediction task called ``text-to-SQL Semantic Parsing'', which translates a natural language question into a structured query language (SQL) program, shows that the low-/high-transparency explanations often lead to less/more user reliance on the model decisions, whereas the medium-transparency explanations strike a good balance.

Abstract

Explaining the decisions of AI has become vital for fostering appropriate user trust in these systems. This paper investigates explanations for a structured prediction task called ``text-to-SQL Semantic Parsing'', which translates a natural language question into a structured query language (SQL) program. In this task setting, we designed three levels of model explanation, each exposing a different amount of the model's decision-making details (called ``algorithm transparency''), and investigated how different model explanations could potentially yield different impacts on the user experience. Our study with 100 participants shows that (1) the low-/high-transparency explanations often lead to less/more user reliance on the model decisions, whereas the medium-transparency explanations strike a good balance. We also show that (2) only the medium-transparency participant group was able to engage further in the interaction and exhibit increasing performance over time, and that (3) they showed the least changes in trust before and after the study.

Paper Structure

This paper contains 42 sections, 1 equation, 14 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Explainable AI and Trust in Human-Machine Interaction
  4. Explaining Text-to-SQL Semantic Parsing
  5. Study Goals
  6. The substantially higher complexity of semantic parsing than simple classification.
  7. Complexity caused by the users' non-technical backgrounds.
  8. The intricate relation between RQ1 and RQ2.
  9. Interface Design for Human Subject Study
  10. Three Levels of Explanations for Text-to-SQL Semantic Parsing
  11. Trust Measurement
  12. Propensity to Trust.
  13. Checklist for Trust between People and Automation (Jian Scale).
  14. Method
  15. Task Setup
  16. ...and 27 more sections

Figures (14)

  • Figure 1: Interface designed to facilitate our human subject study with high-transparency explanation.
  • Figure 2: The "database view" in our interface presents the participants with the database schema and sample table records.
  • Figure 3: A message will pop up after the participant chooses an answer to the question "Do you think this is a correct prediction?".
  • Figure 4: An example explanation at a low transparency level.
  • Figure 5: An example explanation at a medium transparency level.
  • ...and 9 more figures