Situated Brushing and Linking in Virtual and Augmented Reality

TL;DR

This paper investigates brushing and linking in situated analytics by comparing AR and VR in a real-space supermarket scenario using a video see-through display. It evaluates three highlighting techniques—Outline (O), Animated Outline (A), and Animated Outline with Visual Links (L)—across three task types with 40 participants, revealing that AR generally improves time and accuracy for selection tasks, while the effectiveness of techniques depends on condition. Animated linking speeds task completion in VR but can reduce accuracy, and visual links are preferred yet prone to occlusion, highlighting important design trade-offs for integrating digital content with physical referents. The findings provide actionable guidance for AR/VR interface design in situated analytics, including guidance on link routing, animation, and realism constraints when simulating AR in VR. The work also discusses limitations such as realistic fidelity of AR stimuli and device-dependent effects, informing future research on adaptable, clutter-aware highlighting in immersive analytics.

Abstract

In traditional visual analysis, brushing and linking is commonly used to visually connect multiple views using highlighting techniques. However, brushing and linking has rarely been used in situated analytics, which uses visualizations to analyze data in the context of physical referents. In situated analytics, data representations must be visually linked to real-world objects. Previous work has assessed situated brushing and linking in a virtual reality simulation of a supermarket scenario. Here, we replicate and extend the previous approach by studying brushing and linking in an actual physical space with augmented reality, while further improving the highlighting techniques. Using a video see-through display, we compare augmented reality with virtual reality. Results suggest that AR performs better in time and accuracy, but the effectiveness of the techniques varies by condition. These results provide a new framing of how the real-world stimuli matter in situated analytics.

Figures (9)

• Figure 1: We built a small supermarket with three aisles, occupying approximately 25 m². Left: Structure of the supermarket model. Right: Shelf model, with the spatial arrangement for spatial judgment tasks.
• Figure 2: Brushing on a situated scatterplot in our user study. The top row shows the steps to solving a single selection task using O in AR, and the bottom row, a multi-selection task using L in VR. From left to right: First, the user adjusts the data dimensions and filters to meet the task requirements. Second, they brush the data points by selecting one-to-one or delimiting regions. Third, once the data points are selected, linked referents are highlighted. Fourth, the user examines the environment to find the highlighted products, confirming by pointing a ray and pressing the trigger on the controller. Link colors are animated by transitioning from green to orange and back.
• Figure 3: (left) Mean Completion Time (top) and Mean Linking Time (bottom) in seconds for all conditions and tasks. (right) Pairwise differences. Error bars represent 95% bootstrap confidence intervals. Evidence of differences is marked with an asterisk *. The further away from zero and the tighter the CI, the stronger the evidence is.
• Figure 4: (left) Mean error rate in % for selection tasks (top) and for spatial judgment tasks (bottom), for all conditions and tasks. (right) Pairwise differences. Error bars representing 95% bootstrap confidence intervals. Evidence of differences is marked with an asterisk *. The further away from 0% and the tighter the CI, the stronger the evidence is.
• Figure 5: (left) Mean completion time (top) and Mean linking time (bottom) in seconds per technique. (right) Pairwise differences. Error bars represent 95% bootstrap confidence intervals. Evidence of differences is marked with an asterisk *. The further away from zero and the tighter the CI, the stronger the evidence is.