Searching Through Complex Worlds: Visual Search and Spatial Regularity Memory in Mixed Reality

TL;DR

It is found that the secondary auditory task did not have a significant main effect on visual search performance, while significantly elevating higher perceived workload measures in all conditions, and complex environments and varied virtual elements depths significantly hinder visual search, but did not significantly increase perceived workload measures.

Abstract

Visual search is a core component of mixed reality (MR) interactions, influenced by the complexities of MR application contexts. In this paper, we investigate how prevalent factors in MR influence visual search performance and spatial regularity memory -- including the physical environment complexity, secondary task presence, virtual content depth and spatial layout configurations. Contrary to prior work, we found that the secondary auditory task did not have a significant main effect on visual search performance, while significantly elevating higher perceived workload measures in all conditions. Complex environments and varied virtual elements depths significantly hinder visual search, but did not significantly increase perceived workload measures. Finally, participants did not explicitly recognize repeated spatial configurations of virtual elements, but performed significantly better when searching repeated spatial configurations, suggesting implicit memory of spatial regularities. Our work presents novel insights on visual search and highlights key considerations when designing MR for different application contexts.

Figures (9)

• Figure 1: Overview of the contextual cueing task procedure. Each trial displayed a visual search array with one virtual target "T" among eleven virtual distractor "L". Repeated spatial configurations, exemplified at Trial 1 and Trial 101, preserved the same spatial arrangement of distractors, with only the target orientation varied across repetitions. Novel spatial configurations, exemplified at Trial 26, Trial 51, and Trial 76, were newly generated for each trial. The timeline illustrates how repeated and novel spatial configurations appeared across trials. In the experiment, the trial order of repeated and novel spatial configurations was randomized.
• Figure 2: Experimental environments. Left: The simple environment with virtual elements that were presented at the same depth, consisting of a white desk against a white wall, with only a white keyboard on the desk. Right: The complex environment with virtual elements that were presented at different depths, incorporating a monitor, laptop, plush toy, and outdoor background scene visible through a window. While our MR system varied the depth at which virtual elements appeared in the different-depth condition, such variations are difficult to illustrate with 2D images. In 2D, depth manipulations may appear as changes in stimulus size. However, changing size of virtual elements does not have the same effects as changing the depth, as depth variations require users to refocus their eyes to different depth planes deng2021towardslee2024visualtsirlin2016size.
• Figure 3: Overview of the experimental procedure, consisting of a 15-minute briefing session, a 90-minute main experiment composed of eight randomized conditions separated by rests, and a 15-minute debriefing phase with interview and discussion.
• Figure 4: Bar plots for the mean reaction times (ms) of correct trials across three experimental factors: (a) task type, (b) virtual element depth, and (c) physical environment complexity. Statistical significance was assessed using pairwise comparisons of estimated marginal means with Tukey-adjusted p-values. Asterisks denote significant pairwise comparisons (* p < .05, ** p < .01, *** p < .001).
• Figure 5: Bar plots for the mean reaction times (ms) of correct trials across three experimental factors: (a) task type, (b) virtual element depth, and (c) physical environment complexity, examined separately for novel and repeated spatial configurations. Statistical significance was assessed using pairwise comparisons of estimated marginal means with Tukey-adjusted p-values. Asterisks denote significant pairwise comparisons (* p < .05, ** p < .01, *** p < .001).