Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Decoupled Hands: An Approach for Aligning Perspectives in Collaborative Mixed Reality

Matt Gottsacker, Nels Numan, Anthony Steed, Gerd Bruder, Gregory F. Welch, Steve Feiner

TL;DR

This work tackles occlusion and reference-clarity problems in co-located collaborative MR by introducing Decoupled Hands, which aligns collaborators' perspectives around a shared object without moving users. It achieves this by maintaining local copies of the object that are rotated around the object's centroid to match the leader's view, with the leader's hands' representations adapting accordingly to preserve intuitive pointing. The method preserves face-to-face interaction in see-through MR and is demonstrated through a functional Unity-based prototype targeting map-based collaboration, alongside a formal evaluation methodology to compare against other perspective-sharing techniques. The approach promises scalable, exact perspective alignment for large objects and larger collaborator groups, and lays groundwork for extending to physical objects via ML-driven spatial understanding and virtualization.

Abstract

When collaborating relative to a shared 3D virtual object in mixed reality (MR), users may experience communication issues arising from differences in perspective. These issues include occlusion (e.g., one user not being able to see what the other is referring to) and inefficient spatial references (e.g., "to the left of this" may be confusing when users are positioned opposite to each other). This paper presents a novel technique for automatic perspective alignment in collaborative MR involving co-located interaction centered around a shared virtual object. To align one user's perspective on the object with a collaborator's, a local copy of the object and any other virtual elements that reference it (e.g., the collaborator's hands) are dynamically transformed. The technique does not require virtual travel and preserves face-to-face interaction. We created a prototype application to demonstrate our technique and present an evaluation methodology for related MR collaboration and perspective alignment scenarios.

Decoupled Hands: An Approach for Aligning Perspectives in Collaborative Mixed Reality

TL;DR

This work tackles occlusion and reference-clarity problems in co-located collaborative MR by introducing Decoupled Hands, which aligns collaborators' perspectives around a shared object without moving users. It achieves this by maintaining local copies of the object that are rotated around the object's centroid to match the leader's view, with the leader's hands' representations adapting accordingly to preserve intuitive pointing. The method preserves face-to-face interaction in see-through MR and is demonstrated through a functional Unity-based prototype targeting map-based collaboration, alongside a formal evaluation methodology to compare against other perspective-sharing techniques. The approach promises scalable, exact perspective alignment for large objects and larger collaborator groups, and lays groundwork for extending to physical objects via ML-driven spatial understanding and virtualization.

Abstract

When collaborating relative to a shared 3D virtual object in mixed reality (MR), users may experience communication issues arising from differences in perspective. These issues include occlusion (e.g., one user not being able to see what the other is referring to) and inefficient spatial references (e.g., "to the left of this" may be confusing when users are positioned opposite to each other). This paper presents a novel technique for automatic perspective alignment in collaborative MR involving co-located interaction centered around a shared virtual object. To align one user's perspective on the object with a collaborator's, a local copy of the object and any other virtual elements that reference it (e.g., the collaborator's hands) are dynamically transformed. The technique does not require virtual travel and preserves face-to-face interaction. We created a prototype application to demonstrate our technique and present an evaluation methodology for related MR collaboration and perspective alignment scenarios.

Paper Structure

This paper contains 13 sections, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Group Awareness in Collaborative MR
  4. Mixed Reality View Sharing Techniques
  5. Perspective Alignment Technique Design & Implementation
  6. Perpective Alignment Design Principles
  7. Perspective Alignment Technique
  8. Functional Prototype Application
  9. Networking Implementation
  10. Prototype Hardware and Software
  11. MR Object-Based Collaboration Evaluation Methodology
  12. Future Work
  13. Conclusion

Figures (1)

  • Figure 1: Screenshots from our collaborative AR system in use showing different stages of the perspective alignment process. At time t = 1, User A (top panel, orange) points at something that is out of view for User B (bottom panel, blue). At t = 2, User B points at User A and presses the trigger to activate a perspective alignment. At t = 3, the perspective alignment is in progress: User A can see User B's virtual hands rotating around the virtual object toward them; User B can see the virtual object and User A's virtual hands rotating toward them. At t =4, User A has become the View Leader and User B has become the View Follower. Specifically, User B's 3D map has rotated such that he can see the map from the same angle as User A. Each user can see virtual representations of the other user's hands floating over the map in the correct virtual positions relative to the other user's view.