Designing Upper-Body Gesture Interaction with and for People with Spinal Muscular Atrophy in VR

TL;DR

This research conducted an elicitation study in which 12 VR-experienced people with SMA designed upper-body gestures for 26 VR commands, and collected 312 user-defined gestures that tended to enhance their body involvement and preferred gestures that required minimal physical effort, and were aesthetically pleasing.

Abstract

Recent research proposed gaze-assisted gestures to enhance interaction within virtual reality (VR), providing opportunities for people with motor impairments to experience VR. Compared to people with other motor impairments, those with Spinal Muscular Atrophy (SMA) exhibit enhanced distal limb mobility, providing them with more design space. However, it remains unknown what gaze-assisted upper-body gestures people with SMA would want and be able to perform. We conducted an elicitation study in which 12 VR-experienced people with SMA designed upper-body gestures for 26 VR commands, and collected 312 user-defined gestures. Participants predominantly favored creating gestures with their hands. The type of tasks and participants' abilities influence their choice of body parts for gesture design. Participants tended to enhance their body involvement and preferred gestures that required minimal physical effort, and were aesthetically pleasing. Our research will contribute to creating better gesture-based input methods for people with motor impairments to interact with VR.

Figures (10)

• Figure 1: The process for gathering the common VR commands: 1) Category Definition established four categories of VR tasks, named Selection, Manipulation, Menu-based Interaction and Locomotion; 2) Video Collection specified VR tasks from popular VR application videos on YouTube; 3) Video Analysis categorized the VR tasks into the four defined categories, following the established coding rules.
• Figure 2: 30 VR applications in final. There are 32 VR apps in total, but in the Entertainment, Vader Immortal: Episode I, II, and III were regarded as one item because they had the same VR task but different contents.
• Figure 3: The Study Procedure.
• Figure 4: The effect of Grab Distant Object command: (a) demonstrates the video clip frame of Grab Distant Object. One researcher wearing Quest2 recorded first-person views of grabbing a distant object in VR (on the right side of the screen), while another researcher captured his actions in reality (on the left side of the screen). These two videos were edited in time synchronization to assist participants in comprehending VR tasks; (b) depicts the state before and after grabbing, showcasing an author's interaction with the VR system to execute a grabbing motion.
• Figure 5: The Processing of User-defined Gesture Analysis