Evaluating Navigation and Comparison Performance of Computational Notebooks on Desktop and in Virtual Reality

TL;DR

This study investigates converting computational notebooks from desktop to virtual reality (VR), emphasizing navigation and comparison tasks. It introduces a Branching&Merging feature and evaluates four conditions (Desktop/VR × Linear/Branch) via a controlled user study with preconfigured notebooks drawn from scikit-learn. Results show VR substantially improves navigation efficiency and that the Branching mechanism significantly enhances parameter comparisons, while text input in VR remains a major bottleneck. The work provides empirical evidence for the potential of immersive analytics with large spatial displays and embodied interaction, while outlining practical directions to address VR-specific input challenges.

Abstract

The computational notebook serves as a versatile tool for data analysis. However, its conventional user interface falls short of keeping pace with the ever-growing data-related tasks, signaling the need for novel approaches. With the rapid development of interaction techniques and computing environments, there is a growing interest in integrating emerging technologies in data-driven workflows. Virtual reality, in particular, has demonstrated its potential in interactive data visualizations. In this work, we aimed to experiment with adapting computational notebooks into VR and verify the potential benefits VR can bring. We focus on the navigation and comparison aspects as they are primitive components in analysts' workflow. To further improve comparison, we have designed and implemented a Branching&Merging functionality. We tested computational notebooks on the desktop and in VR, both with and without the added Branching&Merging capability. We found VR significantly facilitated navigation compared to desktop, and the ability to create branches enhanced comparison.

Figures (9)

• Figure 1: Visual representation (top) of the Branch&Merge mechanism within computational notebooks, as instantiated in a VR (bottom). Different hues within the results are employed to illustrate the independent storage of variables across branches. The figure is segmented into five key operations: (a) regular notebooks, (b) initial creation of the branch, (c) merge back to a singular code execution path, (d) initiation of the second branch, and (e) final merging process to facilitate value comparisons across all initiated branches.
• Figure 2: Illustrations of gestures interactions in the VR environment.
• Figure 3: Demonstrations of conditions tested in our study. User scrolling to navigate in Desktop (top), and walking and rotating their body to navigate in VR (bottom). In addition, the user used a physical keyboard to write codes in Desktop, while used a virtual keyboard in VR.
• Figure 4: Completion time for VR and Desktop in the navigation task. (a) the time spent for completing deletion, and (b) the time spent completing relocation. Solid lines indicate statistical significance with $p < 0.05$. The tables below show the Cohen's D effect sizes for significant comparisons. Circles with black borders indicate the condition with better results.
• Figure 5: Analysis of four testing conditions in the comparison task. (a) the time spent completing the task, (b) the number of executions performed, (c) the time spent for text input, (d) the time spent completing the task excluding the text input interaction time, (e) the total time spent for creating all branches in Desktop and VR, (f) the average time spent for creating a branch in Desktop and VR. Solid lines indicate statistical significance with $p < 0.05$, and dashed lines indicate $0.05 < p < 0.1$. The tables below show the Cohen's D effect sizes for significant comparisons. Circles with black borders indicate the condition with better results.