PerspectiveCoach: Exploring LLMs for Developer Reflection

TL;DR

PerspectiveCoach investigates whether an LLM-based conversational agent can scaffold structured perspective-taking to deepen developers’ ethical reflection about how software design affects marginalized users. Using a mixed-method evaluation (18 front-end developers) plus a human–human study, the authors analyze reflection depth, perceived usability, and restatement fidelity through four text-similarity metrics and turn-taking analysis. Findings indicate the tool enhances reflection, broadens perspectives, and is generally usable, but faces limitations in adaptability, potential repetition, and power dynamics relative to human conversations. The work contributes exploratory design insights for integrating LLM-based perspective-taking into software practice, emphasizing plurality, context, and practical pathways from reflection to design action.

Abstract

Despite growing awareness of ethical challenges in software development, practitioners still lack structured tools that help them critically engage with the lived experiences of marginalized users. This paper presents PerspectiveCoach, a large language model (LLM)-powered conversational tool designed to guide developers through structured perspective-taking exercises and deepen critical reflection on how software design decisions affect marginalized communities. Through a controlled study with 18 front-end developers (balanced by sex), who interacted with the tool using a real case of online gender-based harassment, we examine how PerspectiveCoach supports ethical reasoning and engagement with user perspectives. Qualitative analysis revealed increased self-awareness, broadened perspectives, and more nuanced ethical articulation, while a complementary human-human study contextualized these findings. Text similarity analyses demonstrated that participants in the human-PerspectiveCoach study improved the fidelity of their restatements over multiple attempts, capturing both surface-level and semantic aspects of user concerns. However, human-PerspectiveCoach's restatements had a lower baseline than the human-human conversations, highlighting contextual differences in impersonal and interpersonal perspective-taking. Across the study, participants rated the tool highly for usability and relevance. This work contributes an exploratory design for LLM-powered end-user perspective-taking that supports critical, ethical self-reflection and offers empirical insights (i.e., enhancing adaptivity, centering plurality) into how such tools can help practitioners build more inclusive and socially responsive technologies.

Figures (3)

• Figure 1: PerspectiveCoach’s configuration interface, where you can define the GPT’s name, description, and behavior. Instructions visible here represent only a subset of the prompt.
• Figure 2: Similarity between participants’ restatements (Attempt 1) and the original user experience across four metrics (TF-IDF cosine, chrF++, ROUGE-L, and Semantic cosine via SBERT). Dashed lines mark the overall median for each metric. Red stars indicate participants who requested revisions to their Attempt 1.
• Figure 3: Mean similarity scores (Attempt 1) for Human–AI vs. Human–Human across four metrics. $\Delta$ above each pair shows $HA - HH$.