Social-Physical Interactions with Virtual Characters: Evaluating the Impact of Physicality through Encountered-Type Haptics

TL;DR

This paper introduces ETHOS, an Encountered-Type Haptic Display that couples a torque-controlled 7-DoF robotic manipulator with interchangeable props and a VR headset to enable social–physical interactions (object handovers, fist bumps, high fives) with a virtual character. Through a mixed factorial study (NP, SP, DP) across three gestures and 54–55 participants, it demonstrates that adding physicality increases presence, realism, enjoyment, and connection, with static props yielding substantial gains and dynamic rendering providing further presence improvements albeit with mixed effects on other metrics. The results highlight the experiential value of embodied touch in VR and suggest that while dynamic strategies hold promise, they require gesture-specific trajectory design and improved social-robot avatar realism to maximize benefits. The work lays a foundation for integrating ETHDs into VR applications such as theatre, rehabilitation, and immersive training, where authentic touch and social connection are central, and points to future directions including adaptive control, compliance-based contact sensing, and enhanced avatar behavior.

Abstract

This work investigates how robot-mediated physicality influences the perception of social-physical interactions with virtual characters. ETHOS (Encountered-Type Haptics for On-demand Social interaction) is an encountered-type haptic display that integrates a torque-controlled manipulator and interchangeable props with a VR headset to enable three gestures: object handovers, fist bumps, and high fives. We conducted a user study to examine how ETHOS adds physicality to virtual character interactions and how this affects presence, realism, enjoyment, and connection metrics. Each participant experienced one interaction under three conditions: no physicality (NP), static physicality (SP), and dynamic physicality (DP). SP extended the purely virtual baseline (NP) by introducing tangible props for direct contact, while DP further incorporated motion and impact forces to emulate natural touch. Results show presence increased stepwise from NP to SP to DP. Realism, enjoyment, and connection also improved with added physicality, though differences between SP and DP were not significant. Comfort remained consistent across conditions, indicating no added psychological friction. These findings demonstrate the experiential value of ETHOS and motivate the integration of encountered-type haptics into socially meaningful VR experiences.

Figures (3)

• Figure 1: Overview of the ETHOS system design. The setup integrates a VR headset for immersive visualization and user hand tracking, ChArUco-based registration to establish a shared anchor between the physical and virtual environments, a robotic manipulator equipped with interchangeable physical props to render on-demand physical contact, and a force sensor to monitor interaction forces.
• Figure 2: Mean ratings of realism, enjoyability, comfort, and character connection across the three physicality conditions (no physicality, static, dynamic). Error bars represent ± standard error of the mean (SEM).
• Figure 3: Presence results visualized across physicality conditions—no physicality, static physicality, and dynamic physicality—for the Multimodal Presence Scale (MPS) makransky2017development: (a) total presence, (b) physical presence, (c) social presence, and (d) self-presence. Significant between-condition differences are annotated (* $p<0.05$, *** $p<0.001$).