HoloDevice: Holographic Cross-Device Interactions for Remote Collaboration

TL;DR

The paper tackles the problem of enabling rich remote collaboration between physical devices and holographically rendered remote counterparts. It introduces holographic cross-device interaction and demonstrates a Hololens 2-based prototype, HoloDevice, with a WebXR-driven rendering and WebSocket-based synchronization. The authors present a taxonomy and a design space centered on three benefits—spatial visualization, intermediate-transition affordances, and dynamic configuration—and illustrate five use cases (content transfer, input/control, edit/annotation, adaptive UI, extended screens) along with ten interaction techniques. The work provides concrete implementation details, including rendering, tracking, and rapid prototyping, and discusses six application scenarios such as remote classrooms and collaborative CAD. Overall, the approach offers a scalable, visually rich framework for remote cross-device collaboration, with clear opportunities for future WebXR integration and design refinement.

Abstract

This paper introduces holographic cross-device interaction, a new class of remote cross-device interactions between local physical devices and holographically rendered remote devices. Cross-device interactions have enabled a rich set of interactions with device ecologies. Most existing research focuses on co-located settings (meaning when users and devices are in the same physical space) to achieve these rich interactions and affordances. In contrast, holographic cross-device interaction allows remote interactions between devices at distant locations by providing a rich visual affordance through real-time holographic rendering of the device's motion, content, and interactions on mixed reality head-mounted displays. This maintains the advantages of having a physical device, such as precise input through touch and pen interaction. Through holographic rendering, not only can remote devices interact as if they are co-located, but they can also be virtually augmented to further enrich interactions, going beyond what is possible with existing cross-device systems. To demonstrate this concept, we developed HoloDevice, a prototype system for holographic cross-device interaction using the Microsoft Hololens 2 augmented reality headset. Our contribution is threefold. First, we introduce the concept of holographic cross-device interaction. Second, we present a design space containing three unique benefits, which include: (1) spatial visualization of interaction and motion, (2) rich visual affordances for intermediate transition, and (3) dynamic and fluid configuration. Last we discuss a set of implementation demonstrations and use-case scenarios that further explore the space.

Figures (8)

• Figure 1: Diagram presents where this work is focused on. Which is the intersection of cross-device interaction, remote collaboration, and holographic mixed reality.
• Figure 2: General overview of holographic cross-device interaction. This system relies on three fundamental parts: the local user, the local physical device, and a holographic rendering of the remote device. The remote user can also be rendered to further support the interactions.
• Figure 3: Different use cases for the holographic cross-device interaction.
• Figure 4: Different unique interactions and layouts from the perspective of the local user.
• Figure 5: User can use the bump gesture to transfer content between devices. The local user moves their device towards the hologram (left). Once the the local phone touches the hologram (middle) the content will be transferred (right).