Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

User-centric evaluation of explainability of AI with and for humans: a comprehensive empirical study

Szymon Bobek, Paloma Korycińska, Monika Krakowska, Maciej Mozolewski, Dorota Rak, Magdalena Zych, Magdalena Wójcik, Grzegorz J. Nalepa

TL;DR

The results of a user-centered assessment of commonly used eXplainable Artificial Intelligence algorithms reveal limitations in existing XAI methods and confirm the need for new design principles and evaluation techniques that address the specific information needs and user perspectives of different classes of AI stakeholders.

Abstract

This study is located in the Human-Centered Artificial Intelligence (HCAI) and focuses on the results of a user-centered assessment of commonly used eXplainable Artificial Intelligence (XAI) algorithms, specifically investigating how humans understand and interact with the explanations provided by these algorithms. To achieve this, we employed a multi-disciplinary approach that included state-of-the-art research methods from social sciences to measure the comprehensibility of explanations generated by a state-of-the-art lachine learning model, specifically the Gradient Boosting Classifier (XGBClassifier). We conducted an extensive empirical user study involving interviews with 39 participants from three different groups, each with varying expertise in data science, data visualization, and domain-specific knowledge related to the dataset used for training the machine learning model. Participants were asked a series of questions to assess their understanding of the model's explanations. To ensure replicability, we built the model using a publicly available dataset from the UC Irvine Machine Learning Repository, focusing on edible and non-edible mushrooms. Our findings reveal limitations in existing XAI methods and confirm the need for new design principles and evaluation techniques that address the specific information needs and user perspectives of different classes of AI stakeholders. We believe that the results of our research and the cross-disciplinary methodology we developed can be successfully adapted to various data types and user profiles, thus promoting dialogue and address opportunities in HCAI research. To support this, we are making the data resulting from our study publicly available.

User-centric evaluation of explainability of AI with and for humans: a comprehensive empirical study

TL;DR

The results of a user-centered assessment of commonly used eXplainable Artificial Intelligence algorithms reveal limitations in existing XAI methods and confirm the need for new design principles and evaluation techniques that address the specific information needs and user perspectives of different classes of AI stakeholders.

Abstract

This study is located in the Human-Centered Artificial Intelligence (HCAI) and focuses on the results of a user-centered assessment of commonly used eXplainable Artificial Intelligence (XAI) algorithms, specifically investigating how humans understand and interact with the explanations provided by these algorithms. To achieve this, we employed a multi-disciplinary approach that included state-of-the-art research methods from social sciences to measure the comprehensibility of explanations generated by a state-of-the-art lachine learning model, specifically the Gradient Boosting Classifier (XGBClassifier). We conducted an extensive empirical user study involving interviews with 39 participants from three different groups, each with varying expertise in data science, data visualization, and domain-specific knowledge related to the dataset used for training the machine learning model. Participants were asked a series of questions to assess their understanding of the model's explanations. To ensure replicability, we built the model using a publicly available dataset from the UC Irvine Machine Learning Repository, focusing on edible and non-edible mushrooms. Our findings reveal limitations in existing XAI methods and confirm the need for new design principles and evaluation techniques that address the specific information needs and user perspectives of different classes of AI stakeholders. We believe that the results of our research and the cross-disciplinary methodology we developed can be successfully adapted to various data types and user profiles, thus promoting dialogue and address opportunities in HCAI research. To support this, we are making the data resulting from our study publicly available.

Paper Structure

This paper contains 13 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Results
  3. Thematic analysis with think-aloud protocol
  4. Reception of the presentation of XAI algorithms
  5. Discussion
  6. Methods
  7. Dataset, AI model and XAI algorithms
  8. Participants
  9. Research tools
  10. Think-aloud protocol (TAP)
  11. Thematic analysis in MAXQDA
  12. Analysis of the user experience
  13. Data availability

Figures (3)

  • Figure 1: Detailed description of the user study
  • Figure 2: Analysis of modifications to the order of explanations and the level of details. The original sequence of slides is presented above the table with modifications suggested by participants.
  • Figure 3: Peirce’s semiotic triangle and its XAI interpretation with object related to instance that is being explained (mushroom in our case), representamen related to the explanation itself (e.g. SHAP values, LIME, etc.) and interpretant representing the comprehension of the sign (i.e the results produced by the AI system) by the user.