Human-centered explanation does not fit all: The interplay of sociotechnical, cognitive, and individual factors in the effect AI explanations in algorithmic decision-making

TL;DR

This study demonstrates that human-centered, purely contrastive or selective explanations are not universally advantageous in AI-assisted decision-making. Using a scenario-based, between-subject survey across six decision contexts and six explanation styles, the authors show that complete explanations are generally valued most, while the effectiveness of contrastive and selective designs strongly depends on context, individual cognitive traits, and prior beliefs. Through nonparametric tests and structural equation modeling, they reveal that germane cognitive load and context-driven motivation/ability interactions chiefly shape explanation valuation, challenging one-size-fits-all XAI strategies. The work offers a novel framework for explanation evaluation that integrates sociotechnical context, cognitive engagement, and individual differences, with practical design implications for personalized AI chatbot interfaces and context-sensitive explainability.

Abstract

Recent XAI studies have investigated what constitutes a \textit{good} explanation in AI-assisted decision-making. Despite the widely accepted human-friendly properties of explanations, such as contrastive and selective, existing studies have yielded inconsistent findings. To address these gaps, our study focuses on the cognitive dimensions of explanation evaluation, by evaluating six explanations with different contrastive strategies and information selectivity and scrutinizing factors behind their valuation process. Our analysis results find that contrastive explanations are not the most preferable or understandable in general; Rather, different contrastive and selective explanations were appreciated to a different extent based on who they are, when, how, and what to explain -- with different level of cognitive load and engagement and sociotechnical contexts. Given these findings, we call for a nuanced view of explanation strategies, with implications for designing AI interfaces to accommodate individual and contextual differences in AI-assisted decision-making.

Figures (12)

• Figure 1: The process of humans' valuation of explanations. This study explores the interplay of individual-, context-, and explanation-dependent factors influencing the valuation of explanation strategies.
• Figure 2: The detailed definitions and examples of six variants of explanations.
• Figure 3: A) Study framework: Our study aims to systematically explore the inner workings of how individuals attribute AI-generated decisions when confronted with AI-driven decisions (e.g., why was I denied a loan?) and evaluate explanations presented by AI systems. The study framework for human valuation on explanations presents a cognitive journey (in the order from a to f) explainees may go through while processing explanations about the decision. B) Survey design: To examine the valuation process, we distilled the framework into a survey for scenario-based experiment. C) AI-assisted decision scenarios: These decision scenarios are carefully selected to be contrasted in three aspects: high-stakes, professional, and timely.
• Figure 4: The relative ratings (along the $x$-axis) for each of the six explanation variants for the extraneous and germane cognitive capacities influenced by how information is presented, as well as the preference rankings in five distinct value dimensions. To facilitate the summary, participants' overall preference was highlighted in gray.
• Figure 5: In each scenario, the relative ratings (along the $x$-axis) for each of the six explanation variants for the extraneous and germane cognitive capacities, as well as the preference rankings in five distinct value dimensions. To facilitate the summary, participants' overall preference was highlighted in gray.