Comparing the Effects of Visual, Haptic, and Visuohaptic Encoding on Memory Retention of Digital Objects in Virtual Reality

TL;DR

It is found that visuohaptic encoding significantly improved memorization accuracy compared to unimodal visual and haptic conditions and its implications for the optimal encoding design in VR applications that assist professionals who need to memorize and recall virtual objects in their daily work.

Abstract

Although Virtual Reality (VR) has undoubtedly improved human interaction with 3D data, users still face difficulties retaining important details of complex digital objects in preparation for physical tasks. To address this issue, we evaluated the potential of visuohaptic integration to improve the memorability of virtual objects in immersive visualizations. In a user study (N=20), participants performed a delayed match-to-sample task where they memorized stimuli of visual, haptic, or visuohaptic encoding conditions. We assessed performance differences between these encoding modalities through error rates and response times. We found that visuohaptic encoding significantly improved memorization accuracy compared to unimodal visual and haptic conditions. Our analysis indicates that integrating haptics into immersive visualizations enhances the memorability of digital objects. We discuss its implications for the optimal encoding design in VR applications that assist professionals who need to memorize and recall virtual objects in their daily work.

Figures (7)

• Figure 1: A participant using VR and a force-feedback device during an experimental trial.
• Figure 2: Illustration of the study procedure. After providing written informed consent and demographics, participants were trained on the task and accommodated with the VR environment and haptic feedback. Participants completed six trial runs containing five trials grouped in single-condition blocks, where the order of conditions was randomized across participants. Cells with denote the visual condition, while describe haptic and are visuohaptic conditions.
• Figure 3: Sample stimulus (left) and its corresponding Foil stimulus (right) used in a 2AFC. The stimuli are 3D 5x5 matrices with continuous paths of 8 connected beveled blocks and one beveled block outside the path. The foil stimulus differs from the sample by a one-block change either to the path or the single modified block. Each trial presented a unique Sample/Foil pair for each trial.
• Figure 4: Summary of the experiment. Six experimental runs were divided into three counterbalanced single-condition blocks, each containing five trials of its condition. Each trial was divided into: 1) the encoding phase, 7 seconds; 2) the delay: 5 seconds; and 3) the testing phase: up to 14 seconds.
• Figure 5: (a): A participant moves the virtual probe towards a sample during the learning phase. (b): A mask reveals the stimulus at the touched location. (c): Sample and foil exploration during the testing phase.