ClipGS-VR: Immersive and Interactive Cinematic Visualization of Volumetric Medical Data in Mobile Virtual Reality
Yuqi Tong, Ruiyang Li, Chengkun Li, Qixuan Liu, Shi Qiu, Pheng-Ann Heng
TL;DR
This work tackles the challenge of high-fidelity cinematic visualization of volumetric medical data on mobile VR by enabling arbitrary-angle slicing. It introduces ClipGS-VR, which precomputes and consolidates high-fidelity layers from 200 slicing states into a unified rendering structure to avoid runtime neural inference and run in real time on standalone headsets. A gradient-based opacity modulation, expressed as $\alpha' = \alpha \cdot \sigma$ with $\sigma = \text{clamp}(\tfrac{1}{2} + (\boldsymbol{\mu} \cdot \mathbf{n} - c)/(2 \cdot s_{\mathbf{n}}), 0, 1)$, ensures smooth, artifact-free intersections for arbitrary slicing. Evaluations show fidelity comparable to offline results while delivering improved usability and interaction efficiency, with a user study confirming higher usability scores and reduced fatigue on 6-DoF slicing in VR.
Abstract
High-fidelity cinematic medical visualization on mobile virtual reality (VR) remains challenging. Although ClipGS enables cross-sectional exploration via 3D Gaussian Splatting, it lacks arbitrary-angle slicing on consumer-grade VR headsets. To achieve real-time interactive performance, we introduce ClipGS-VR and restructure ClipGS's neural inference into a consolidated dataset, integrating high-fidelity layers from multiple pre-computed slicing states into a unified rendering structure. Our framework further supports arbitrary-angle slicing via gradient-based opacity modulation for smooth, visually coherent rendering. Evaluations confirm our approach maintains visual fidelity comparable to offline results while offering superior usability and interaction efficiency.