User Characteristics in Explainable AI: The Rabbit Hole of Personalization?

TL;DR

This work provides evidence to reorient user-focused XAI research and question the pursuit of personalized XAI based on fine-grained user characteristics, as very few user characteristics mattered; only age and the personality trait openness influenced actual understanding.

Abstract

As Artificial Intelligence (AI) becomes ubiquitous, the need for Explainable AI (XAI) has become critical for transparency and trust among users. A significant challenge in XAI is catering to diverse users, such as data scientists, domain experts, and end-users. Recent research has started to investigate how users' characteristics impact interactions with and user experience of explanations, with a view to personalizing XAI. However, are we heading down a rabbit hole by focusing on unimportant details? Our research aimed to investigate how user characteristics are related to using, understanding, and trusting an AI system that provides explanations. Our empirical study with 149 participants who interacted with an XAI system that flagged inappropriate comments showed that very few user characteristics mattered; only age and the personality trait openness influenced actual understanding. Our work provides evidence to reorient user-focused XAI research and question the pursuit of personalized XAI based on fine-grained user characteristics.

Figures (2)

• Figure 1: The prototype interface. (A) A list of all the comments to be assessed. (B) The selected comment and a bar chart representing the top 10 most important words in the selected comment are sorted in descending order. (C) Ability to add new important words that are not already highlighted. (D) A summary of the AI system's prediction for the selected comment and the ability to change the predicted label. (E) A list of the top 10 most important words in the selected comment. For each important word, the user can change the label (which label, Toxic/Non-toxic, is most associated with the word) and the word importance (how important the word is in the prediction).
• Figure 2: Our user study procedure consists of six steps. Step 1 - Study introduction: The participants were provided with an overview of the study's objectives, the potential outcomes of their involvement, the utilization of their data, as well as the advantages and disadvantages of participation. Step 2 - Pre-task questionnaire: The participants completed a pre-task questionnaire consisting of two sections to measure their personality and previous experience with AI systems. Step 3 - Task introduction: Participants were presented with a scenario and an introduction to the task that they were required to complete for the prototype. Step 4 - Practice task: Participants were instructed to complete a practice task using the explanatory debugging interface. Step 5 - Main task: The primary task involved the utilization of the explanatory debugging interface to complete the study task. Step 6 - Post-task questionnaire: Participants were requested to fill out a post-task questionnaire to measure the participant's trust, perceived understanding, and actual understanding.