AI Systems in Text-Based Online Counselling: Ethical Considerations Across Three Implementation Approaches
Philipp Steigerwald, Jennifer Burghardt, Eric Rudolph, Jens Albrecht
TL;DR
This paper addresses the ethical governance of AI in text-based online counselling (TBOC) across three implementation approaches: autonomous counsellor bots, AI counsellee simulators, and counsellor-facing augmentation tools. It proposes a taxonomy of these approaches and grounds ethical analysis in four core principles—privacy, fairness, autonomy, and accountability—applying them to each approach to reveal distinct risk profiles and governance needs. The work highlights that autonomous bots entail the highest risk with respect to crisis response and competence boundaries, simulators raise concerns about ecological validity and representational fairness, and augmentation tools introduce issues of invisible AI mediation and automation bias. The overarching message is that AI can augment TBOC at scale, but responsible, human-centered deployment and ongoing governance are essential to preserve counselling’s core values and safety.
Abstract
Text-based online counselling scales across geographical and stigma barriers, yet faces practitioner shortages, lacks non-verbal cues and suffers inconsistent quality assurance. Whilst artificial intelligence offers promising solutions, its use in mental health counselling raises distinct ethical challenges. This paper analyses three AI implementation approaches - autonomous counsellor bots, AI training simulators and counsellor-facing augmentation tools. Drawing on professional codes, regulatory frameworks and scholarly literature, we identify four ethical principles - privacy, fairness, autonomy and accountability - and demonstrate their distinct manifestations across implementation approaches. Textual constraints may enable AI integration whilst requiring attention to implementation-specific hazards. This conceptual paper sensitises developers, researchers and practitioners to navigate AI-enhanced counselling ethics whilst preserving human values central to mental health support.