Visual Guidance for User Placement in Avatar-Mediated Telepresence between Dissimilar Spaces

TL;DR

This work addresses the challenge of preserving a local user’s interaction context (gaze and pointing) in avatar-mediated telepresence across dissimilar spaces. It defines an angle-based interaction feature and a Gaussian similarity measure to score how well local placements can be mirrored by remote avatar placements, selecting an Optimal Corresponding Placement via a cost-based optimization. Visual guidance is provided through color-coded 2D sectors and overlays of remote-space models, with validation in VR and MR experiments showing that the scores align with user perception and that the guidance improves placement decisions. The approach offers a practical method for enhancing bidirectional MR telepresence in real-world scenarios where local and remote spaces differ in layout and furniture.

Abstract

Rapid advances in technology gradually realize immersive mixed-reality (MR) telepresence between distant spaces. This paper presents a novel visual guidance system for avatar-mediated telepresence, directing users to optimal placements that facilitate the clear transfer of gaze and pointing contexts through remote avatars in dissimilar spaces, where the spatial relationship between the remote avatar and the interaction targets may differ from that of the local user. Representing the spatial relationship between the user/avatar and interaction targets with angle-based interaction features, we assign recommendation scores of sampled local placements as their maximum feature similarity with remote placements. These scores are visualized as color-coded 2D sectors to inform the users of better placements for interaction with selected targets. In addition, virtual objects of the remote space are overlapped with the local space for the user to better understand the recommendations. We examine whether the proposed score measure agrees with the actual user perception of the partner's interaction context and find a score threshold for recommendation through user experiments in virtual reality (VR). A subsequent user study in VR investigates the effectiveness and perceptual overload of different combinations of visualizations. Finally, we conduct a user study in an MR telepresence scenario to evaluate the effectiveness of our method in real-world applications.

Figures (15)

• Figure 1: User egocentric (left) and room perspective (right) views of space A (user X) and space B (user Y) in our MR telepresence system. Virtual avatars X' and Y' appear in remote spaces (space B for X', space A for Y') to represent user X and Y, respectively. When user X selects interaction targets (TV and Y' in this example) to interact with, our system provides visual guidance, color-coded sectors on the floor, and transparent 3D models of the remote space, to assist X in selecting his placement that will allow for his avatar X' to be appropriately placed to interact with the remote corresponding targets (TV and user Y in space B). After X arrives at his selected placement, our system places avatar X’ at an optimal location that best corresponds to user X's placement in space A, allowing bidirectional interaction between X and Y through their avatars.
• Figure 2: An example placement problem in MR telepresence between dissimilar spaces. In space A, both placements $X_1$ and $X_2$ suitably accommodate the interaction between the user and the target object (screen). However, their corresponding avatar placements that have the identical spatial relation between the avatar and the target object show that $X_2'$ is inappropriate due to the collision with a table, making $X_1$ a better placement than $X_2$. However, the users cannot predict the quality of the avatar placement that their placement will bring without additional information. This limitation emphasizes the need for supplementary guidance to enhance users' understanding and decision-making processes concerning avatar placement in MR telepresence scenarios.
• Figure 3: Four angles that build the interaction feature between a source object (a user, $s$) and a target object (a screen, $t$). Green rectangles on objects represent the bounding boxes for defining endpoints.
• Figure 4: (a) Interactions features for two placements with the same direction but different distance from the target $(x_t,y_t)$. (b) Two placements with the same features $\phi_{s \rightarrow t}$ for targets with different sizes.
• Figure 5: Feature similarities of sample placements of the avatar X', shown as arrows. The interaction target is set as Y for X', and Y' for X. The green color corresponds to a similarity of 1.00, and the smaller the similarity, the closer the color is to black.