Learning High-Quality Navigation and Zooming on Omnidirectional Images in Virtual Reality

TL;DR

OmniVR addresses blur in VR-viewed ODIs by learning a high-fidelity refinement that is directed by user navigation and zoom commands. The method maps (beta,gamma,s) from headset and controller inputs to a Möbius transformation, applies a learning-based, HR-space refinement with spherical indexing and Slerp-based resampling, and outputs perspective-projected ODIs for VR display. The approach demonstrates state-of-the-art performance on the ODIM dataset and yields positive user-study results, showing improved recognition, reduced discomfort, and heightened immersion. Overall, OmniVR enables intuitive navigation and zoom with enhanced visual clarity, offering practical impact for VR media consumption and interactive ODI applications.

Abstract

Viewing omnidirectional images (ODIs) in virtual reality (VR) represents a novel form of media that provides immersive experiences for users to navigate and interact with digital content. Nonetheless, this sense of immersion can be greatly compromised by a blur effect that masks details and hampers the user's ability to engage with objects of interest. In this paper, we present a novel system, called OmniVR, designed to enhance visual clarity during VR navigation. Our system enables users to effortlessly locate and zoom in on the objects of interest in VR. It captures user commands for navigation and zoom, converting these inputs into parameters for the Mobius transformation matrix. Leveraging these parameters, the ODI is refined using a learning-based algorithm. The resultant ODI is presented within the VR media, effectively reducing blur and increasing user engagement. To verify the effectiveness of our system, we first evaluate our algorithm with state-of-the-art methods on public datasets, which achieves the best performance. Furthermore, we undertake a comprehensive user study to evaluate viewer experiences across diverse scenarios and to gather their qualitative feedback from multiple perspectives. The outcomes reveal that our system enhances user engagement by improving the viewers' recognition, reducing discomfort, and improving the overall immersive experience. Our system makes the navigation and zoom more user-friendly.

Figures (13)

• Figure 1: Comparing the VR experience under the baseline condition and our OmniVR. Users can freely navigate and zoom in/out to see the object of interest. With our proposed algorithm, the objects can be refined with clear textural details, thus enhancing the engagement and immersive experience.
• Figure 2: System overview. Our system collects the user commands for navigation and zooming in/out, which are converted to the parameters of the Möbius transformation matrix. The parameters together with the ODI are processed with a learning-based algorithm to generate high-quality transformed ODIs, which can be displayed with the perspective format in VR.
• Figure 3: The rotation of the VR headset generates horizontal and vertical angle parameters, while the trigger button of the right controller is used for generating zoom level parameters.
• Figure 4: The overall pipeline of the proposed algorithm. With the spatial index generation module and spherical resampling module, OmniVR can provide viewers with a flexible way to zoom in and out to objects of interest, such as the sculpture.
• Figure 5: The illustration of the proposed spatial index generation module. With the HR feature map as input, the spatial index generation module generates a transformed index map to accomplish the feature transformation process.