Better Together? The Role of Explanations in Supporting Novices in Individual and Collective Deliberations about AI

TL;DR

This study addresses how explanations in XAI can support AI novices in both individual and group deliberations about deploying AI in public institutions. It tests a modular, question-driven explanation design across four information categories using a task-based interview study with 43 participants (8 focus groups and 12 interviews) around an AMS employment prediction use case. Findings show that groups foster shared understanding and argumentation, while individuals achieve higher task performance but miss collaborative exchange; both settings address different facets of understanding. The authors offer design guidelines to optimize explanations for group settings, discuss real-world contexts such as public forums and workplaces, and highlight the need to balance information load with social dynamics to support deliberation on public AI systems.

Abstract

Deploying AI systems in public institutions can have far-reaching consequences for many people, making it a matter of public interest. Providing opportunities for stakeholders to come together, understand these systems, and debate their merits and harms is thus essential. Explainable AI often focuses on individuals, but deliberation benefits from group settings, which are underexplored. To address this gap, we present findings from an interview study with 8 focus groups and 12 individuals. Our findings provide insight into how explanations support AI novices in deliberating alone and in groups. Participants used modular explanations with four information categories to solve tasks and decide about an AI system's deployment. We found that the explanations supported groups in creating shared understanding and in finding arguments for and against the system's deployment. In comparison, individual participants engaged with explanations in more depth and performed better in the study tasks, but missed an exchange with others. Based on our findings, we provide suggestions on how explanations should be designed to work in group settings and describe their potential use in real-world contexts. With this, our contributions inform XAI research that aims to enable AI novices to understand and deliberate AI systems in the public sector.

Figures (15)

• Figure 1: Overview of explanations. Explanations were designed as a collection of 36 question-answer pairs. The questions were assigned to 4 categories, data, system details, usage, and context, each containing 9 questions. Participants received the base explanations at the beginning of the explanation phase, as indicated by the ticked boxes, and could request all other explanations at any time during the explanation phase using this overview.
• Figure 2: Four explanation examples. Examples for explanations in the categories data, system details, usage, and context. Each question was printed on a sheet of A5 paper with a short answer to the question. Answers could be fully textual or complemented with visual elements like charts or colored shapes. Each category was given a different color and icon to facilitate navigation.
• Figure 3: Overview of the study procedure. Focus groups and single interviews differed only in the explanation phase and the questions for the second individual reports.
• Figure 4: Material for individual reports of participants. Participants received the materials for individual reports on laminated paper slips in different colors (blue, yellow, red, green, and grey) and used them to answer questions individually. Slips that were numbered with letters a to f (A) served as 5-point scales for understanding, confidence, inclusion in group, and influence of group discussion and were answered by writing a number (1--5); slips with icons and a corresponding textual description (B) served as selection of the most helpful and influential explanation categories and were answered by selecting any number of icons; slips with decisions (C) served as voting ballots for deployment decisions and were answered by ticking yes or no.
• Figure 5: Both explanation and social dynamic have an impact on collaborative performance. In focus groups, both explanations and social dynamic were key factors for the understanding outcome. If participants could engage easily with the explanations and each other, their interactions realized mechanisms of 'collaborative success' nokes-malach_when_2015 and led to shared understanding. In contrast, if participants had trouble using the explanations and could not outsource or discuss these issues, interactions rather realized mechanisms of 'failure' nokes-malach_when_2015 (Section \ref{['sec:analysis']}) and showed impeded understanding. Depending on these intermediary steps, groups experienced the outcomes as working or abandoned understanding. From the perspective of XAI, both explanation and social dynamic are thus important aspects to keep in mind when designing explanations for groups in collaborative settings.