Large Language Models in Peer-Run Community Behavioral Health Services: Understanding Peer Specialists and Service Users' Perspectives on Opportunities, Risks, and Mitigation Strategies

TL;DR

This study investigates how large language models (LLMs) can be integrated into peer-run behavioral health services without compromising the relational authority that underpins recovery-oriented peer support. Leveraging comicboarding with Collaborative Support Programs of New Jersey (CSPNJ), the authors gathered insights from 16 peer specialists and 10 service users, revealing three core tensions: bridging scale with locality, protecting trust, and preserving peer autonomy. The work contributes a lived-experience-in-the-loop design principle, recasts trust as co-constructed, and frames LLMs as relational collaborators rather than clinical arbiters. These findings offer actionable guidance for building community-centered AI that enhances peer support while safeguarding relational integrity and autonomy, with implications for broader care contexts where trust and human relationships are central.

Abstract

Peer-run organizations (PROs) provide critical, recovery-based behavioral health support rooted in lived experience. As large language models (LLMs) enter this domain, their scale, conversationality, and opacity introduce new challenges for situatedness, trust, and autonomy. Partnering with Collaborative Support Programs of New Jersey (CSPNJ), a statewide PRO in the Northeastern United States, we used comicboarding, a co-design method, to conduct workshops with 16 peer specialists and 10 service users exploring perceptions of integrating an LLM-based recommendation system into peer support. Findings show that depending on how LLMs are introduced, constrained, and co-used, they can reconfigure in-room dynamics by sustaining, undermining, or amplifying the relational authority that grounds peer support. We identify opportunities, risks, and mitigation strategies across three tensions: bridging scale and locality, protecting trust and relational dynamics, and preserving peer autonomy amid efficiency gains. We contribute design implications that center lived-experience-in-the-loop, reframe trust as co-constructed, and position LLMs not as clinical tools but as relational collaborators in high-stakes, community-led care.

Figures (7)

• Figure 1: Method and study process using LLM-Lifecycle Comicboards. Participants review an introductory comicboard explaining what an LLM system is (see Appendix Figure \ref{['fig:comicboard0']}), familiarize themselves with two personas, then engage with three stages of the LLM lifecycle: (1) what the LLM system can do (see Appendix Figure \ref{['fig:comicboard1']}), (2) what it needs (see Appendix Figure \ref{['fig:comicboard2']}), and (3) how it gathers feedback to improve (see Appendix Figure \ref{['fig:comicboard3']}). At each stage, participants read scenario-based comicboards, reflect on potential opportunities and risks, suggest mitigation strategies, and comment on the research team’s proposed mitigation solutions (see Appendix Figure \ref{['fig:comicboard4']}).
• Figure 2: Findings overview illustrating the crosscutting core theme of "relational authority" and its manifestation across three tensions shaping LLM integration in peer-run behavioral health services: (1) Scale vs. Situatedness, (2) Trust vs. Mistrust, and (3) Autonomy vs. Automation, alongside participant-identified opportunities, risks, and mitigation strategies that influence whether LLM use sustains, undermines, or amplifies relational authority.
• Figure 3: Introduction comicboards (“What is an LLM?”) illustrating the definition of LLMs and four key aspects of the LLM development process: pre-training, supervised fine-tuning, reinforcement learning, and prompting.
• Figure 4: Stage 1 comicboards (“What does the LLM tool do?”) depicting how an LLM assists peer specialists in gathering key information during sessions, organizing it, and providing personalized recommendations for wellness goals, resources, and benefit eligibility, to support timely and context-aware peer support in peer-run behavioral health settings.
• Figure 5: Stage 2 comicboards (“What does the LLM tool need?”) illustrating how an LLM gathers and organizes information from peer specialists’ notes and session transcripts, displays recommendations clearly, and enables real-time updates or corrections in peer-run behavioral health settings.