Traversing Dual Realities: Investigating Techniques for Transitioning 3D Objects between Desktop and Augmented Reality Environments

TL;DR

The work investigates how to move 3D objects between a desktop monitor and AR space, proposing three baseline transition techniques and evaluating them through two studies (an initial usability test with eighteen participants and an expert study with six computational chemists). Using a Unity-based prototype with both desktop and AR components, the authors reveal that hand-gesture–driven transitions are generally preferred, with animation aiding perception during long-distance moves, and they derive design guidance for multi-technique, distance-enabled desktop+AR workflows. The results demonstrate practical benefits for cross-reality workflows in 3D visualization and domain tasks while highlighting device-imperfections and user-adaptation challenges, informing future cross-reality system design. The research thus advances practical, embodied methods for transitioning digital objects across realities, with implications for scientific visualization, collaboration, and knowledge-work in hybrid desktop+AR environments.

Abstract

Desktop environments can integrate augmented reality (AR) head-worn devices to support 3D representations, visualizations, and interactions in a novel yet familiar setting. As users navigate across the dual realities -- desktop and AR -- a way to move 3D objects between them is needed. We devise three baseline transition techniques based on common approaches in the literature and evaluate their usability and practicality in an initial user study (N=18). After refining both our transition techniques and the surrounding technical setup, we validate the applicability of the overall concept for real-world activities in an expert user study (N=6). In it, computational chemists followed their usual desktop workflows to build, manipulate, and analyze 3D molecular structures, but now aided with the addition of AR and our transition techniques. Based on our findings from both user studies, we provide lessons learned and takeaways for the design of 3D object transition techniques in desktop + AR environments.

Figures (5)

• Figure 1: Schematic of a transition process. First, the user has to decide which object to transition and make a selection accordingly. Second, a transition has to be triggered. Then, a transition is supported by an animation ($t=0$ to $t=N$) until the end state is reached.
• Figure 2: The setup for our initial study. The participants sat in front of a monitor wearing an HMD. A grid of objects is presented in the desktop environment, and a similar grid is shown in AR next to the participants. Image transformed using Stable Diffusion.
• Figure 3: Top: 95% confidence intervals of the six individual NASA-TLX subscales and the total Raw-TLX score for the transition techniques. Bottom: 95% confidence intervals of Likert scale ratings for the effectiveness of the transition techniques with and without animation.
• Figure 4: A: Task completion times. B: The average number of incorrect triggers performed by each participant per trial and their 95% confidence intervals. C: The average number of redundant triggers performed by each participant per trial and their 95% confidence intervals.
• Figure 5: Our refined techniques with the categories "Button Press Alternative" (Hover+Button and Gaze+Button), "New Techniques" (Distant Pull and Flick), and "Distant Grab Alternative" (Catch and Throw). Image transformed using Stable Diffusion.