Studying Mobile Spatial Collaboration across Video Calls and Augmented Reality

TL;DR

The paper addresses the gap in understanding how mobile video and mobile AR support spatial collaboration at a distance. It uses a comparative structured observation with a dual-mode prototype to study fourteen pairs performing spatial investigations, analyzed with mixed methods including video, transcripts, motion logs, maps, and interviews. The contributions include a detailed account of how medium choice shapes collaborator roles, the shared language for coordination, and embodied interaction, along with design implications for integrating video and AR in future mobile collaboration tools. The findings indicate that video helps participants stay 'on the same page' via a common visual pane, while AR enables embodied, space-aware collaboration that more closely mirrors in-person interaction, suggesting a hybrid approach as a practical path forward.

Abstract

Mobile video calls are widely used to share information about real-world objects and environments with remote collaborators. While these calls provide valuable visual context in real time, the experience of interacting with people and moving around a space is significantly reduced when compared to co-located conversations. Recent work has demonstrated the potential of Mobile Augmented Reality applications to enable more spatial forms of collaboration across distance. To better understand the dynamics of mobile AR collaboration and how this medium compares against the status quo, we conducted a comparative structured observation study to analyze people's perception of space and interaction with remote collaborators across mobile video calls and AR-based calls. Fourteen pairs of participants completed a spatial collaboration task using each medium. Through a mixed-methods analysis of session videos, transcripts, motion logs, post-task exercises, and interviews, we highlight how the choice of medium influences the roles and responsibilities that collaborators take on and the construction of a shared language for coordination. We discuss the importance of spatial reasoning with one's body, how video calls help participants "be on the same page" more directly, and how AR calls enable both onsite and remote collaborators to engage with the space and each other in ways that resemble in-person interaction. Our study offers a nuanced view of the benefits and limitations of both mediums, and we conclude with a discussion of design implications for future systems that integrate mobile video and AR to better support spatial collaboration in its many forms.

Figures (12)

• Figure 1: An overview of the spatial collaboration study, using Session 9 as an example. (Top) The AR Call. Onsite Participant 9 (OP9, center-left) is in the real space, and can view Remote Participant 9 (RP9), as a moving video feed in AR (left). RP9 is in a different room (center-right), and can view both the moving video of OP9, and a 3D model of the real space in AR. (Bottom) The video call. Here too, OP9 is in the real space (center-left), and uses their phone to share the video of the room with RP9 who is in a different room (center-right).
• Figure 2: An overview of the AR Call mode. Elements viewed in AR through the phone are colored blue. (Left) The onsite participant is in the real space, and can view the moving video representation of the remote participant in AR through their phone. (Right) The remote participant uses their phone to view a pre-scanned 3D model of the real space, as well as the moving video representation of the onsite participant.
• Figure 3: Photographs of the two spatial investigation scenes used in the study, with a corresponding top-down sketch of the space and the various clues spread across the rooms.
• Figure 4: Video call interaction from Session 2. (Left) OP2 using gestures to call attention to objects in the space, while consciously framing the camera view. (Right) RP2 collaborating from a different room.
• Figure 5: Video call interaction from Session 3. (Left) OP3 is exploring the space and narrating their thoughts while holding the phone to the side. (Center) As a result, both their self-view video and the environment video are not framed properly in the shared feed. (Right) RP3 does not have a proper view into the space, and cannot engage in face-to-face communication.