Decoupled Edge Physics algorithms for collaborative XR simulations

TL;DR

We address the challenge of delivering realistic, interactive XR simulations at scale on untethered HMDs by decoupling the physics engine from the rendering pipeline and offloading it to edge/cloud infrastructure using a central $N-1$ architecture. The proposed GHost-PhyS framework, along with PCC/PO-based entity replication and a lightweight Riptide-based communication layer, enables real-time multi-user interactions with up to 100 collaborators and thousands of physics objects, including soft bodies, while preserving single-player QoE. Key contributions include a dual-representation scene graph (GrO/PO), a robust session-initiation protocol for multi-user setups, a collision and interaction system, softbody simulation on the PhyS, and a relay/network-optimization strategy with dual-quaternion interpolation to smooth updates. Experimental results demonstrate substantial reductions in HMD frame times, sustained high PhyS throughput, and favorable network performance, validating the practicality of edge-cloud physics for immersive, collaborative XR workloads. The work has practical impact by enabling high-fidelity XR on untethered devices, reducing local CPU load, and expanding support for large-scale multi-user collaboration in VR/AR environments.

Abstract

This work proposes a novel approach to transform any modern game engine pipeline, for optimized performance and enhanced user experiences in Extended Reality (XR) environments. Decoupling the physics engine from the game engine pipeline and using a client-server N-1 architecture creates a scalable solution, efficiently serving multiple graphics clients on Head-Mounted Displays (HMDs) with a single physics engine on edge-cloud infrastructure. This approach ensures better synchronization in multiplayer scenarios without introducing overhead in single-player experiences, maintaining session continuity despite changes in user participation. Relocating the Physics Engine to an edge or cloud node reduces strain on local hardware, dedicating more resources to high-quality rendering and unlocking the full potential of untethered HMDs. We present four algorithms that decouple the physics engine, increasing frame rates and Quality of Experience (QoE) in VR simulations, supporting advanced interactions, numerous physics objects, and multi-user sessions with over 100 concurrent users. Incorporating a Geometric Algebra interpolator reduces inter-calls between dissected parts, maintaining QoE and easing network stress. Experimental validation, with more than 100 concurrent users, 10,000 physics objects, and softbody simulations, confirms the technical viability of the proposed architecture, showcasing transformative capabilities for more immersive and collaborative XR applications without compromising performance.

Figures (16)

• Figure 1: Amphitheatrically placed 100 VR users around the patient
• Figure 2: Top: Overview of the N-1 Decoupled approach. Bottom: Process of dissecting an entity
• Figure 3: Top: PhyS Initialization process. Bot-Left: Hand Colliders. Sphere Collider bellow the hand is the Trigger Collider. Bot-Right: Soft Body Bunny with particles and spring connections between them
• Figure 4: Top: Architecture with Relay Server and PhyS. Bottom: Softbody Experimentation Scenario.
• Figure 5: Top: 100 CCU experimentation with HMD view and Physics objects in PhyS. Bottom: MultiObject Experimentation Scenario.