AI, Help Me Think$\unicode{x2014}$but for Myself: Assisting People in Complex Decision-Making by Providing Different Kinds of Cognitive Support

TL;DR

This paper investigates how large language models can augment human decision-making beyond traditional recommendations by introducing ExtendAI, an AI that extends the user’s forward reasoning, and compares it to a RecommendAI that provides direct recommendations. Through a mixed-methods study in a realistic ETF investing task with $N=21$ participants, ExtendAI demonstrated stronger integration with users’ thinking and slightly better outcomes, while RecommendAI offered novel insights but required more cognitive effort. The study identifies three core tensions in AI-assisted decision-making: actionability versus cognitive engagement, novelty versus alignment with user reasoning, and timing of AI input within the decision process. The findings suggest a design space between recommendation-centric AI and reasoning-augmentation approaches, with implications for trust calibration, user agency, and practical deployment in complex domains.

Abstract

How can we design AI tools that effectively support human decision-making by complementing and enhancing users' reasoning processes? Common recommendation-centric approaches face challenges such as inappropriate reliance or a lack of integration with users' decision-making processes. Here, we explore an alternative interaction model in which the AI outputs build upon users' own decision-making rationales. We compare this approach, which we call ExtendAI, with a recommendation-based AI. Participants in our mixed-methods user study interacted with both AIs as part of an investment decision-making task. We found that the AIs had different impacts, with ExtendAI integrating better into the decision-making process and people's own thinking and leading to slightly better outcomes. RecommendAI was able to provide more novel insights while requiring less cognitive effort. We discuss the implications of these and other findings along with three tensions of AI-assisted decision-making which our study revealed.

Figures (6)

• Figure 1: Screenshot of the simulated ETF trading platform interface used in the study, showcasing the various components including the ETF list, portfolio overview, ETF details, investor profile, and the investing assistant.
• Figure 2: A schematic overview of the ETF trading platform, illustrating what participants did and what the AI assistant panel showed in each of the three steps of the study: familiarisation phase, RecommendAI, and ExtendAI.
• Figure 3: Participants' scores on a 5-point scale for the questions on AI suggestion consideration, insights, helpfulness, interference with decision-making, likelihood of usage, and trust. R-AI refers to RecommendAI and E-AI to ExtendAI.
• Figure 4: Distribution of Nasa-TLX raw scores indicating cognitive effort for ExtendAI (E-AI) and RecommendAI (R-AI).
• Figure 5: Participants' scores on a 5-point scale on the extent of feeling informed, their confidence in their decisions, and their satisfaction with decision outcomes for the familiarisation phase (FAM) RecommendAI (R-AI) and ExtendAI (E-AI). FAM is greyed out for Satisfaction, as it cannot be meaningfully interpreted.