Toward Safe Evolution of Artificial Intelligence (AI) based Conversational Agents to Support Adolescent Mental and Sexual Health Knowledge Discovery

TL;DR

The paper addresses the safety challenges of AI-based conversational agents used by adolescents for mental and sexual health knowledge discovery. It surveys the current landscape, distinguishing rule-based and LLM-based CAs, and articulates key safety concerns such as restricted, incoherent, or inappropriate content. The authors advocate for guardrails, adolescent-inclusive design, and rigorous safety evaluations to guide the safe evolution of child-centered AI in health contexts. This position piece aims to stimulate dialogue and collaboration at the intersection of AI technology, child-centered design, and clinical support to enable safer, more effective adolescent health information discovery via conversational systems.

Abstract

Following the recent release of various Artificial Intelligence (AI) based Conversation Agents (CAs), adolescents are increasingly using CAs for interactive knowledge discovery on sensitive topics, including mental and sexual health topics. Exploring such sensitive topics through online search has been an essential part of adolescent development, and CAs can support their knowledge discovery on such topics through human-like dialogues. Yet, unintended risks have been documented with adolescents' interactions with AI-based CAs, such as being exposed to inappropriate content, false information, and/or being given advice that is detrimental to their mental and physical well-being (e.g., to self-harm). In this position paper, we discuss the current landscape and opportunities for CAs to support adolescents' mental and sexual health knowledge discovery. We also discuss some of the challenges related to ensuring the safety of adolescents when interacting with CAs regarding sexual and mental health topics. We call for a discourse on how to set guardrails for the safe evolution of AI-based CAs for adolescents.