Stretch your reach: Studying Self-Avatar and Controller Misalignment in Virtual Reality Interaction

TL;DR

The study addresses misalignment between a full-body self-avatar and real controllers in VR by evaluating five interaction metaphors that vary rendering, attachment, and arm stretch. Using a within-subjects design with Cube, Cannon, and Painting tasks and 40 participants, it measures embodiment, proprioception, preference, and performance. The key finding is that stretching the avatar's arms to align end-effectors (StretchController/StretchHand) yields the strongest embodiment, proprioceptive accuracy, user preference, and task performance, while rendering the controller has limited impact. These results yield practical guidelines for designing manipulation techniques with full-body avatars, emphasizing end-effector alignment through arm stretch to enhance user experience. The average maximum arm stretch observed was 26.3 cm (SD 4.65 cm).

Abstract

Immersive Virtual Reality typically requires a head-mounted display (HMD) to visualize the environment and hand-held controllers to interact with the virtual objects. Recently, many applications display full-body avatars to represent the user and animate the arms to follow the controllers. Embodiment is higher when the self-avatar movements align correctly with the user. However, having a full-body self-avatar following the user's movements can be challenging due to the disparities between the virtual body and the user's body. This can lead to misalignments in the hand position that can be noticeable when interacting with virtual objects. In this work, we propose five different interaction modes to allow the user to interact with virtual objects despite the self-avatar and controller misalignment and study their influence on embodiment, proprioception, preference, and task performance. We modify aspects such as whether the virtual controllers are rendered, whether controllers are rendered in their real physical location or attached to the user's hand, and whether stretching the avatar arms to always reach the real controllers. We evaluate the interaction modes both quantitatively (performance metrics) and qualitatively (embodiment, proprioception, and user preference questionnaires). Our results show that the stretching arms solution, which provides body continuity and guarantees that the virtual hands or controllers are in the correct location, offers the best results in embodiment, user preference, proprioception, and performance. Also, rendering the controller does not have an effect on either embodiment or user preference.

Figures (9)

• Figure 1: Illustration of the different factors in two scenarios. The top row represents the scenario in which the factor is enabled, and the bottom row corresponds to the disabled factor. From left to right, the factors $\mathit{Controller}$, $\mathit{Attached}$, and $\mathit{Stretch}$ are shown. In situations where the virtual controller is not aligned with the real one, the real controller is depicted in the figure with a lighter color (not in the simulation).
• Figure 2: Diagram of the user study protocol. The experiment begins with participants being embodied in a virtual avatar, animated using the AvatarGo library Ponton:2022. Participants then perform three tasks (in randomized order) in the virtual environment and answer an embodiment questionnaire for each condition. In the subsequent stage, participants answer a question regarding their perception of the positions of their real hands and virtual hands. Finally, after being introduced to the interaction modes, participants explore all conditions freely and provide feedback via the final preference questionnaire.
• Figure 3: In the preference questionnaire phase, participants could freely switch between interaction methods using the VR controller's touchpad interfaced with a virtual watch display (left), aiding users in visualizing their selected mode. To simplify the recognition of each interaction mode, they were depicted by a unique icon, consistent with its representation in the preference questionnaire (right).
• Figure 4: Proportional representation of responses to the question LQ1Do you think your hand was in the position of your virtual hand? ($\mathit{Proprioception}$) across different interaction modes ($\mathit{Mode}$). Each mode is represented by a stacked bar, indicating the proportion of Yes (1) and No (0) responses.
• Figure 5: Hypothesis H2. One-way repeated measures ANOVA on ranks (Friedman test) of $\mathit{Mode}$ on $\mathit{Embodiment}$ and the corresponding post-hoc tests (Wilcoxon signed-rank test). $p$ is the adjusted p-value with Bonferroni correction. Effect Size is Kendall's W for the Friedman test and the $r$ value for the post-hoc tests.