Locomotion in CAVE: Enhancing Immersion through Full-Body Motion

TL;DR

This paper addresses the challenge of awkward locomotion in CAVE systems by introducing a markerless full‑body locomotion framework that leverages dynamic PnP calibration and real‑time action recognition to map user movements to virtual navigation. It presents a three‑layer CAVE architecture, a calibration method, and a SlowFast‑based action recognition pipeline that runs in real time to drive rendering. A dedicated, multi‑camera dataset and a multi‑stage processing pipeline underlie robust posture estimation and action classification, with performance optimizations to handle occlusion and dynamics. A user study demonstrates reduced simulator sickness and enhanced self‑presence, indicating that embodied locomotion can significantly improve immersion and comfort in CAVE environments, though it introduces modest cognitive load and has practical limitations to address in future work.

Abstract

Cave Automatic Virtual Environment (CAVE) is one of the virtual reality (VR) immersive devices currently used to present virtual environments. However, the locomotion methods in the CAVE are limited by unnatural interaction methods, severely hindering the user experience and immersion in the CAVE. We proposed a locomotion framework for CAVE environments aimed at enhancing the immersive locomotion experience through optimized human motion recognition technology. Firstly, we construct a four-sided display CAVE system, then through the dynamic method based on Perspective-n-Point to calibrate the camera, using the obtained camera intrinsics and extrinsic parameters, and an action recognition architecture to get the action category. At last, transform the action category to a graphical workstation that renders display effects on the screen. We designed a user study to validate the effectiveness of our method. Compared to the traditional methods, our method has significant improvements in realness and self-presence in the virtual environment, effectively reducing motion sickness.

Figures (9)

• Figure 1: User study scenario. (a) The participant is navigating the scene in CAVE with controller-based locomotion; (b) the participant is navigating the scene in CAVE with Embodied CAVE Locomotion.
• Figure 2: Overall System Architecture: CL denotes the Computing Layer; HL denotes the Hardware Layer; TL denotes the Transmission Layer.
• Figure 3: The X-Y-Z coordinate system is generated from the corner points of the Aruco markers.
• Figure 4: Overview of the Real-Time Action Recognition and Orientation Detection System framework. The system first performs human detection and tracking on the images, then reconstructs 3D skeleton sequences from the 2D keypoint coordinates. Finally, a high-performance action recognition network analyzes these 3D skeleton sequences and outputs the recognized action category and user ID.
• Figure 5: The actions in the dataset: moving left, standing still, and moving right.