Testing Human-Robot Interaction in Virtual Reality: Experience from a Study on Speech Act Classification

TL;DR

The paper investigates how immersive and non-immersive virtual reality (VR) experiences affect Human-Robot Interaction (HRI) in a smart agriculture setting by focusing on speech-act understanding in a table-grape vineyard. It presents a ROS-based speech-act pipeline trained on an Italian utterance dataset, and compares immersive versus non-immersive VR in two user studies (IVE and NIVE) with 81 participants. Findings show that immersive VR generally enhances engagement, presence, immersion, and related metrics, though space and fatigue considerations favor non-immersive setups for longer studies; a hybrid approach is recommended for CANOPIES tasks. The work provides practical guidance for selecting VR modalities in HRI experiments in precision agriculture and informs data collection and evaluation strategies for future CANOPIES deployments.

Abstract

In recent years, an increasing number of Human-Robot Interaction (HRI) approaches have been implemented and evaluated in Virtual Reality (VR), as it allows to speed-up design iterations and makes it safer for the final user to evaluate and master the HRI primitives. However, identifying the most suitable VR experience is not straightforward. In this work, we evaluate how, in a smart agriculture scenario, immersive and non-immersive VR are perceived by users with respect to a speech act understanding task. In particular, we collect opinions and suggestions from the 81 participants involved in both experiments to highlight the strengths and weaknesses of these different experiences.

Figures (2)

• Figure 1: Developed ROS speech pipeline.
• Figure 2: Graphic representation of the questionnaire responses.