Exploring the Impact of Explainable AI and Cognitive Capabilities on Users' Decisions

TL;DR

This study investigates how AI information (prediction, confidence, accuracy) and multiple explanation styles (example-based, feature-based, rule-based, counterfactual) affect decision quality, reliance, and cognitive load in a loan-approval task, and whether NFC moderates these effects. Using a large online study (N=$288$) with six AI assistance conditions and NFC-based grouping, the authors find that high AI confidence increases reliance on AI and reduces cognitive load, while feature-based explanations do not consistently improve accuracy. Counterfactual explanations, though sometimes less understandable, can enhance overall accuracy and reduce cognitive load when AI predictions are correct, suggesting benefits to hybrid XAI designs. NFC did not reliably differentiate performance or cognitive load in this complex task, pointing to the need to explore additional user characteristics and personalized, context-aware AI interfaces. The results emphasize user-centric XAI design that integrates multiple explanation styles and calibrates AI information to optimize human-AI collaboration in high-stakes decisions.

Abstract

Artificial Intelligence (AI) systems are increasingly used for decision-making across domains, raising debates over the information and explanations they should provide. Most research on Explainable AI (XAI) has focused on feature-based explanations, with less attention on alternative styles. Personality traits like the Need for Cognition (NFC) can also lead to different decision-making outcomes among low and high NFC individuals. We investigated how presenting AI information (prediction, confidence, and accuracy) and different explanation styles (example-based, feature-based, rule-based, and counterfactual) affect accuracy, reliance on AI, and cognitive load in a loan application scenario. We also examined low and high NFC individuals' differences in prioritizing XAI interface elements (loan attributes, AI information, and explanations), accuracy, and cognitive load. Our findings show that high AI confidence significantly increases reliance on AI while reducing cognitive load. Feature-based explanations did not enhance accuracy compared to other conditions. Although counterfactual explanations were less understandable, they enhanced overall accuracy, increasing reliance on AI and reducing cognitive load when AI predictions were correct. Both low and high NFC individuals prioritized explanations after loan attributes, leaving AI information as the least important. However, we found no significant differences between low and high NFC groups in accuracy or cognitive load, raising questions about the role of personality traits in AI-assisted decision-making. These findings highlight the need for user-centric personalization in XAI interfaces, incorporating diverse explanation styles and exploring multiple personality traits and other user characteristics to optimize human-AI collaboration.

Figures (7)

• Figure 1: AI assistance conditions for the loan approval tasks. Participants can display additional information about the attributes by hovering over the info buttons. (A - No AI) Participants will see the task's goal and the current applicant's details. (B - AI) Participants will also be assisted by an AI in the decision-making task (i.e., with prediction, confidence, and accuracy). (C - Example-based) Participants will see condition "B - AI" and the three nearest neighbors of the current applicant. (D - Feature-based) Participants will see condition "B - AI" and the Shapley feature contribution for each applicant's attribute. (E - Rule-based) Participants will see condition "B - AI" and the rule generated by Anchor. (F - Counterfactual) Participants will see condition "B - AI" and three counterfactual instances generated by DiCE.
• Figure 2: Illustration of the procedure participants engaged in during our study.
• Figure 3: Effects of low and high AI confidence considering reliance on AI (H1a), cognitive load (H1b) (ticks above bars indicate lower and higher confidence intervals based on standard errors) and users' accuracy (H1c) divided by AI assistance conditions. The asterisks highlight p-value significance strength (***$p$$<$ .001).
• Figure 4: XAI interface components rank frequencies for low (H2a) and high (H2b) NFC individuals. The asterisks highlight p-value significant strength (***$p$$<$ .001).
• Figure 5: Users' accuracy (H3a) and cognitive load (H3b) disaggregated by low and high NFC (ticks above bars indicate the Standard Error).