Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Metacognitive Demands and Strategies While Using Off-The-Shelf AI Conversational Agents for Health Information

Shri Harini Ramesh, Foroozan Daneshzand, Babak Rashidi, Shriti Raj, Hariharan Subramonyam, Fateme Rajabiyazdi

TL;DR

This paper investigates how users experience metacognitive demands when using off-the-shelf AI conversational agents for health information seeking. Using a think-aloud study with 15 participants and clinician-validated scenarios, it maps the metacognitive demands to the GenAI–human workflow across prompt formulation, evaluation, iteration, and workflow adaptation, and identifies coping strategies. The findings motivate design recommendations to reduce cognitive load, improve transparency, and support safer, more effective health information seeking, including scaffolds for goals, structured prompting, disclosure safeguards, input transparency, and visual/reflection aids. The work contributes practical guidelines for moving beyond generic AI interfaces toward health information systems tailored to metacognitive needs with tangible implications for user experience and safety in health contexts.

Abstract

As Artificial Intelligence (AI) conversational agents become widespread, people are increasingly using them for health information seeking. The use of off-the-shelf conversational agents for health information seeking could place high metacognitive demands (the need for extensive monitoring and control of one's own thought process) on individuals, which could compromise their experience of seeking health information. However, currently, the specific demands that arise while using conversational agents for health information seeking, and the strategies people use to cope with those demands, remain unknown. To address these gaps, we conducted a think-aloud study with 15 participants as they sought health information using our off-the-shelf AI conversational agent. We identified the metacognitive demands such systems impose, the strategies people adopt in response, and propose considerations for designing beyond off-the-shelf interfaces to reduce these demands and support better user experiences and affordances in health information seeking.

Metacognitive Demands and Strategies While Using Off-The-Shelf AI Conversational Agents for Health Information

TL;DR

This paper investigates how users experience metacognitive demands when using off-the-shelf AI conversational agents for health information seeking. Using a think-aloud study with 15 participants and clinician-validated scenarios, it maps the metacognitive demands to the GenAI–human workflow across prompt formulation, evaluation, iteration, and workflow adaptation, and identifies coping strategies. The findings motivate design recommendations to reduce cognitive load, improve transparency, and support safer, more effective health information seeking, including scaffolds for goals, structured prompting, disclosure safeguards, input transparency, and visual/reflection aids. The work contributes practical guidelines for moving beyond generic AI interfaces toward health information systems tailored to metacognitive needs with tangible implications for user experience and safety in health contexts.

Abstract

As Artificial Intelligence (AI) conversational agents become widespread, people are increasingly using them for health information seeking. The use of off-the-shelf conversational agents for health information seeking could place high metacognitive demands (the need for extensive monitoring and control of one's own thought process) on individuals, which could compromise their experience of seeking health information. However, currently, the specific demands that arise while using conversational agents for health information seeking, and the strategies people use to cope with those demands, remain unknown. To address these gaps, we conducted a think-aloud study with 15 participants as they sought health information using our off-the-shelf AI conversational agent. We identified the metacognitive demands such systems impose, the strategies people adopt in response, and propose considerations for designing beyond off-the-shelf interfaces to reduce these demands and support better user experiences and affordances in health information seeking.
Paper Structure (19 sections, 2 figures, 1 table)

This paper contains 19 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Health Information Seeking
  4. Challenges and Metacognitive Demands with AI conversational agents
  5. Systems and Strategies Supporting Health Information Seeking with AI Conversational Agents
  6. Methods
  7. Health Information Seeking Scenarios Development
  8. Participant Sampling and Recruitment
  9. Data collection
  10. Data analysis
  11. Results
  12. Prompt formulation: self-awareness, task decomposition, and well-adjusted confidence
  13. Evaluation: well-adjusted confidence
  14. Prompt Iteration: well-adjusted confidence and metacognitive flexibility
  15. Understanding and Adapting the Workflow: self-awareness, well-adjusted confidence, and metacognitive flexibility
  16. ...and 4 more sections

Figures (2)

  • Figure 1: Screenshot of the AI conversational agent interface we developed to study the metacognitive demands encountered while gathering health information. This image shows an interaction from one of our participants during the study.
  • Figure 2: Summary of the metacognitive demands placed on people using AI conversational agents for health information seeking. Inspired by prior work on metacognitive demands Tankelevitch2024 and empirically observed in our study, the figure highlights the demands that were posed on people across health information seeking goals, zooming in on one goal as an illustration. Specifically, metacognition demands raised by AI agents during (a) prompt formulation, (b) evaluation, (c) prompt iteration, (d) understanding the workflow, and (e) adapting the workflow.