Accessible Nonverbal Cues to Support Conversations in VR for Blind and Low Vision People

Abstract

Social VR has increased in popularity due to its affordances for rich, embodied, and nonverbal communication. However, nonverbal communication remains inaccessible for blind and low vision people in social VR. We designed accessible cues with audio and haptics to represent three nonverbal behaviors: eye contact, head shaking, and head nodding. We evaluated these cues in real-time conversation tasks where 16 blind and low vision participants conversed with two other users in VR. We found that the cues were effective in supporting conversations in VR. Participants had statistically significantly higher scores for accuracy and confidence in detecting attention during conversations with the cues than without. We also found that participants had a range of preferences and uses for the cues, such as learning social norms. We present design implications for handling additional cues in the future, such as the challenges of incorporating AI. Through this work, we take a step towards making interpersonal embodied interactions in VR fully accessible for blind and low vision people.

Figures (4)

• Figure 1: The virtual environments used in the study. Left: the tutorial room, designed to look like a small waiting room. Right: the task room, designed to look like a professional conference room. The same room was used for both baseline and treatment tasks.
• Figure 2: Histograms comparing each participants’ accuracy and confidence scores between the task without cues and the task with cues. Red indicates worse scores with cues, yellow indicates equal scores, and green indicates better scores with cues. Gray indicates one or more of the answers were not provided by participants (NA) so a comparison cannot be made.
• Figure 3: Likert-scale score responses for cue usefulness and distraction.
• Figure 4: Likert-scale score responses for task ease with cues and without cues.