Principles of Safe AI Companions for Youth: Parent and Expert Perspectives

TL;DR

This paper investigates how parents and developmental psychology experts perceive risks and safeguards for youth interactions with GenAI companions. Using 26 semi-structured interviews and real youth–AI conversation snippets, it reveals context-dependent risk assessments: parents tend toward event-based judgments, while experts favor pattern-based monitoring that emphasizes developmental skill-building. The study proposes a multilayered safeguard framework spanning system/character design (ratings, transparency, AI literacy, lobby safety), interaction-level safeguards (contextual monitoring, soft stops, reflective prompts, educational dialogue, disclaimers), and social interventions (crisis-focused parental involvement, resources, and structured notifications). The findings advocate for personalized, transparent, and co-regulated governance that respects youth autonomy and family values, guiding safer design and governance of youth-oriented AI companions.

Abstract

AI companions are increasingly popular among teenagers, yet current platforms lack safeguards to address developmental risks and harmful normalization. Despite growing concerns, little is known about how parents and developmental psychology experts assess these interactions or what protections they consider necessary. We conducted 26 semi structured interviews with parents and experts, who reviewed real world youth GenAI companion conversation snippets. We found that stakeholders assessed risks contextually, attending to factors such as youth maturity, AI character age, and how AI characters modeled values and norms. We also identified distinct logics of assessment: parents flagged single events, such as a mention of suicide or flirtation, as high risk, whereas experts looked for patterns over time, such as repeated references to self harm or sustained dependence. Both groups proposed interventions, with parents favoring broader oversight and experts preferring cautious, crisis-only escalation paired with youth facing safeguards. These findings provide directions for embedding safety into AI companion design.