When Do We Feel Present in a Virtual Reality? Towards Sensitivity and User Acceptance of Presence Questionnaires

TL;DR

This work tackles the challenge of measuring presence in VR by evaluating four popular questionnaires ($SUS$, $IPQ$, $PQ$, $SIP$) for dimension-specific sensitivity across five presence dimensions ($PI$, $PSI$, $INV$, $EMB$, $SOC$). Using five VR scenarios designed to isolate each dimension and a within-subject design with two intensity conditions per dimension, the authors demonstrate that questionnaire sensitivity is highly dimension- and context-dependent: SUS and IPQ are most sensitive to Place Illusion, PQ to Plausibility, and SIP shows broad sensitivity but with high variance. They also reveal a mismatch between score differences and participants' perceived differences, with IPQ best aligning to expectations while SIP often over- or under-estimates changes. The study concludes that questionnaire choice can bias presence assessments and offers practical, dimension-focused guidelines to improve validity and interpretability in VR presence research, while calling for refinement of items and guidance from developers.

Abstract

Presence is an important and widely used metric to measure the quality of virtual reality (VR) applications. Given the multifaceted and subjective nature of presence, the most common measures for presence are questionnaires. But there is little research on their validity regarding specific presence dimensions and their responsiveness to differences in perception among users. We investigated four presence questionnaires (SUS, PQ, IPQ, Bouchard) on their responsiveness to intensity variations of known presence dimensions and asked users about their consistency with their experience. Therefore, we created five VR scenarios that were designed to emphasize a specific presence dimension. Our findings showed heterogeneous sensitivity of the questionnaires dependent on the different dimensions of presence. This highlights a context-specific suitability of presence questionnaires. The questionnaires' sensitivity was further stated as lower than actually perceived. Based on our findings, we offer guidance on selecting these questionnaires based on their suitability for particular use cases.

Figures (15)

• Figure 1: Overview of the presence dimensions that we chose for our investigations split by the two presence models considered. On the right: Dimensions of the presence model by Skarbez et al. (2017, p.23), which summarizes previous models based on a literature review skarbez_survey_2017. On the left: Dimensions of the presence model by Hein et al. (2018, p.10), which summarizes the presence models of Slater and Schubert et al. Hein2018TheUO.
• Figure 2: Example of a paired bar chart showing the scores of positive and negative condition. The score of each bar was shown above the bar. A radio button list on the right allowed users to state their approval.
• Figure 3: Left: Dimension of the real room and the object placed in it. Middle & right: Photos of the study room and the interaction space from the right back corner and the right front corner of the room. a) The cabinet that is recreated in the virtual scenario, b) The scented candle that creates the olfactory stimuli, c) The fan that creates the haptic stimuli, d) The doors that are placed in the same position in the virtual scenario.
• Figure 4: The virtual environments of the place illusion scenario. On the left is the low-dimension intensity condition, and on the right is the high-dimension intensity condition. On the bottom right is a section of the real-world room from the same perspective.
• Figure 5: The plausibility scenario (a), the involvement scenario (b), and the social presence scenario (c).