Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Framing Responsible Design of AI Mental Well-Being Support: AI as Primary Care, Nutritional Supplement, or Yoga Instructor?

Ned Cooper, Jose A. Guridi, Angel Hsing-Chi Hwang, Beth Kolko, Beth McGinty, Qian Yang

TL;DR

The paper tackles how to design non-clinical LLM tools for mental well-being in a responsible, actionable way amid rising adoption and uncertain risks. Through a three-stage, mixed-methods study—expert interviews, policy analysis, and deliberations with diverse experts—it articulates three core criteria for responsibility: defined, guaranteed benefits for specific users; guaranteed delivery of proven active ingredients; and commensurate risk–benefit balancing. It further introduces four analogies (nutritional supplements, OTC drugs, yoga instructors, and primary care providers) to frame responsibilities and regulatory considerations, supported by a stakeholder-informed evaluation framework. The work yields practical design actions, identifies design-research opportunities (active-ingredient databases, primary-care design patterns), and outlines responsible AI research directions (alternative evaluation paradigms beyond population-level metrics). Overall, the study offers a concrete, analogy-based framework to guide safer, more accountable deployment of AI-enabled mental well-being tools at scale, while inviting ongoing debate about evaluation standards and governance.

Abstract

Millions of people now use non-clinical Large Language Model (LLM) tools like ChatGPT for mental well-being support. This paper investigates what it means to design such tools responsibly, and how to operationalize that responsibility in their design and evaluation. By interviewing experts and analyzing related regulations, we found that designing an LLM tool responsibly involves: (1) Articulating the specific benefits it guarantees and for whom. Does it guarantee specific, proven relief, like an over-the-counter drug, or offer minimal guarantees, like a nutritional supplement? (2) Specifying the LLM tool's "active ingredients" for improving well-being and whether it guarantees their effective delivery (like a primary care provider) or not (like a yoga instructor). These specifications outline an LLM tool's pertinent risks, appropriate evaluation metrics, and the respective responsibilities of LLM developers, tool designers, and users. These analogies - LLM tools as supplements, drugs, yoga instructors, and primary care providers - can scaffold further conversations about their responsible design.

Framing Responsible Design of AI Mental Well-Being Support: AI as Primary Care, Nutritional Supplement, or Yoga Instructor?

TL;DR

The paper tackles how to design non-clinical LLM tools for mental well-being in a responsible, actionable way amid rising adoption and uncertain risks. Through a three-stage, mixed-methods study—expert interviews, policy analysis, and deliberations with diverse experts—it articulates three core criteria for responsibility: defined, guaranteed benefits for specific users; guaranteed delivery of proven active ingredients; and commensurate risk–benefit balancing. It further introduces four analogies (nutritional supplements, OTC drugs, yoga instructors, and primary care providers) to frame responsibilities and regulatory considerations, supported by a stakeholder-informed evaluation framework. The work yields practical design actions, identifies design-research opportunities (active-ingredient databases, primary-care design patterns), and outlines responsible AI research directions (alternative evaluation paradigms beyond population-level metrics). Overall, the study offers a concrete, analogy-based framework to guide safer, more accountable deployment of AI-enabled mental well-being tools at scale, while inviting ongoing debate about evaluation standards and governance.

Abstract

Millions of people now use non-clinical Large Language Model (LLM) tools like ChatGPT for mental well-being support. This paper investigates what it means to design such tools responsibly, and how to operationalize that responsibility in their design and evaluation. By interviewing experts and analyzing related regulations, we found that designing an LLM tool responsibly involves: (1) Articulating the specific benefits it guarantees and for whom. Does it guarantee specific, proven relief, like an over-the-counter drug, or offer minimal guarantees, like a nutritional supplement? (2) Specifying the LLM tool's "active ingredients" for improving well-being and whether it guarantees their effective delivery (like a primary care provider) or not (like a yoga instructor). These specifications outline an LLM tool's pertinent risks, appropriate evaluation metrics, and the respective responsibilities of LLM developers, tool designers, and users. These analogies - LLM tools as supplements, drugs, yoga instructors, and primary care providers - can scaffold further conversations about their responsible design.
Paper Structure (37 sections, 2 figures, 4 tables)

This paper contains 37 sections, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLM Mental Well-Being Support in Research
  4. Industries' Attempts to Design AI Mental Well-Being Support Responsibly
  5. Method
  6. Stage 1: Expert Interviews
  7. Stage 2: Policy Analysis
  8. Stage 3: Deliberating Findings with Experts
  9. Findings
  10. Guaranteed Benefits for Intended Users
  11. Guaranteed Delivery of Active Ingredients
  12. Commensurate Risks and Benefits
  13. Discussion
  14. Designing LLM Tools More Responsibly
  15. Design Research Opportunities
  16. ...and 22 more sections

Figures (2)

  • Figure 1: Our findings distinguish four types of non-clinical LLM tools based on their guaranteed benefits. This distinction has important implications for understanding what it means to design a non-clinical LLM tool responsibly.
  • Figure 2: Our findings distinguish four types of non-clinical LLM tools based on their guaranteed benefits. This distinction has important implications for understanding what it means to design a non-clinical LLM tool responsibly.