Designing for Harm Reduction: Communication Repair for Multicultural Users' Voice Interactions

TL;DR

This paper tackles persistent quality-of-service harms faced by multicultural voice assistant users, showing how high speech-recognition errors propagate downstream harms that affect identity, safety, and emotional wellbeing. It follows a psychology-informed approach to design harm-reducing conversational repairs, grounded in interviews and design activities with 16 multicultural participants. The authors identify six downstream harms, illuminate how users personify assistants, and offer concrete guidelines for intermittent identity affirmations, cultural sensitivity, and blame redirection. The work advances a harm-reduction framework for voice interfaces and suggests extensions to other AI technologies, with potential to improve inclusivity and user trust in diverse populations.

Abstract

Voice assistants' inability to serve people-of-color and non-native English speakers has largely been documented as a quality-of-service harm. However, little work has investigated what downstream harms propagate from this poor service. How does poor usability materially manifest and affect users' lives? And what interaction designs might help users recover from these effects? We identify 6 downstream harms that propagate from quality-of-service harms in voice assistants. Through interviews and design activities with 16 multicultural participants, we unveil these 6 harms, outline how multicultural users uniquely personify their voice assistant, and suggest how these harms and personifications may affect their interactions. Lastly, we employ techniques from psychology on communication repair to contribute suggestions for harm-reducing repair that may be implemented in voice technologies. Our communication repair strategies include: identity affirmations (intermittent frequency), cultural sensitivity, and blame redirection. This work shows potential for a harm-repair framework to positively influence voice interactions.

Figures (3)

• Figure 1: Participants often envisioned their assistant as being low-tech. P15 was one of the few participants who incorporated technology in their conceptualization of their assistant, and he did so by including a large, outdated "analog call center." The lefthand stick figure represents P15, and the righthand figure represents Siri. To process requests, Siri must use the button-operated machine. Siri is faced away from the P15 "because it doesn't really take into consideration what I was doing." A fruit fly circles around Siri, demonstrating how Siri may be distracted and perform poorly.
• Figure 2: Participants perceived their assistant as lacking emotional and social skills. This manifested either as (a) lack of emotion or (b) possessing fake emotions and overstepping communication norms.
• Figure 3: Some participants were weary of the corporate ties assistants had. P3 alludes to Apple Inc, by drawing Siri with an apple head. Similar to P16's assistant (Figure \ref{['fig:P16']}), Siri's eyes are covered to denote the assistant's insincerity and non-human elements megias2020reading. Siri's clothing is a gothic business style, alluding to sinister corporate behaviors and data practices.