Investigating Search Among Physical and Virtual Objects Under Different Lighting Conditions
You-Jin Kim, Radha Kumaran, Ehsan Sayyad, Anne Milner, Tom Bullock, Barry Giesbrecht, Tobias Höllerer
TL;DR
This study investigates how lighting and cognitive load affect search behavior in a wide-area outdoor AR task using the HoloLens-2. It employs a 3-factor design (task type, gem location, lighting) with 48 participants performing a gem-search/classification task while occasionally monitoring an auditory stream, in both natural and night lighting in an outdoor courtyard. Key findings show that natural daylight increases perceived task difficulty and collision risk, that virtual content receives more attention than physical content, and that dual-tasking shifts search behavior by reducing head movements, with notable effects contingent on lighting. The results underscore the need to consider perceptual and cognitive constraints, along with occlusion handling and tracking robustness, for realizing effective anywhere-and-anytime mobile AR in outdoor environments.
Abstract
By situating computer-generated content in the physical world, mobile augmented reality (AR) can support many tasks that involve effective search and inspection of physical environments. Currently, there is limited information regarding the viability of using AR in realistic wide-area outdoor environments and how AR experiences affect human behavior in these environments. Here, we conducted a wide-area outdoor AR user study (n = 48) using a commercially available AR headset (Microsoft Hololens 2) to compare (1) user interactions with physical and virtual objects in the environment (2) the effects of different lighting conditions on user behavior and AR experience and (3) the impact of varying cognitive load on AR task performance. Participants engaged in a treasure hunt task where they searched for and classified virtual target items (green ``gems") in an augmented outdoor courtyard scene populated with physical and virtual objects. Cognitive load was manipulated so that in half the search trials users were required to monitor an audio stream and respond to specific target sounds. Walking paths, head orientation and eye gaze information were measured, and users were queried about their memory of encountered objects and provided feedback on the experience. Key findings included (1) Participants self-reported significantly lower comfort in the ambient natural light condition, with virtual objects more visible and participants more likely to walk into physical objects at night; (2) recall for physical objects was worse than for virtual objects, (3) participants discovered more gems hidden behind virtual objects than physical objects, implying higher attention on virtual objects and (4) dual-tasking modified search behavior. These results suggest there are important technical, perceptual and cognitive factors that must be considered.