Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mesh2SLAM in VR: A Fast Geometry-Based SLAM Framework for Rapid Prototyping in Virtual Reality Applications

Carlos Augusto Pinheiro de Sousa, Heiko Hamann, Oliver Deussen

TL;DR

Mesh2SLAM tackles the challenge of prototyping SLAM concepts in VR under limited sensor access and compute on HMDs. It introduces a geometry-based, vertex-feature SLAM that projects mesh vertices to the image plane and uses vertex IDs as descriptors, running in two SLAM threads with a GPU-accelerated front-end. The approach achieves high efficiency and competitive accuracy, outperforming image-feature-based baselines and enabling standalone VR SLAM on low-cost HMDs. This work provides a practical tool for XR localization experiments and rapid SLAM prototyping, with future directions toward multi-agent localization.

Abstract

SLAM is a foundational technique with broad applications in robotics and AR/VR. SLAM simulations evaluate new concepts, but testing on resource-constrained devices, such as VR HMDs, faces challenges: high computational cost and restricted sensor data access. This work proposes a sparse framework using mesh geometry projections as features, which improves efficiency and circumvents direct sensor data access, advancing SLAM research as we demonstrate in VR and through numerical evaluation.

Mesh2SLAM in VR: A Fast Geometry-Based SLAM Framework for Rapid Prototyping in Virtual Reality Applications

TL;DR

Mesh2SLAM tackles the challenge of prototyping SLAM concepts in VR under limited sensor access and compute on HMDs. It introduces a geometry-based, vertex-feature SLAM that projects mesh vertices to the image plane and uses vertex IDs as descriptors, running in two SLAM threads with a GPU-accelerated front-end. The approach achieves high efficiency and competitive accuracy, outperforming image-feature-based baselines and enabling standalone VR SLAM on low-cost HMDs. This work provides a practical tool for XR localization experiments and rapid SLAM prototyping, with future directions toward multi-agent localization.

Abstract

SLAM is a foundational technique with broad applications in robotics and AR/VR. SLAM simulations evaluate new concepts, but testing on resource-constrained devices, such as VR HMDs, faces challenges: high computational cost and restricted sensor data access. This work proposes a sparse framework using mesh geometry projections as features, which improves efficiency and circumvents direct sensor data access, advancing SLAM research as we demonstrate in VR and through numerical evaluation.
Paper Structure (18 sections, 6 equations, 8 figures, 2 tables)

This paper contains 18 sections, 6 equations, 8 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Overview
  5. Frame Capture
  6. Feature Proccesing with vertex features
  7. SLAM Loop
  8. Viewer
  9. OpenXR Integration
  10. Experiments and Results
  11. Efficiency of Vertex Feature Extraction
  12. System Performance
  13. Experiment Setup
  14. Absolute Trajectory Error Visual Inspection
  15. Virtual Reality Practical Evaluation
  16. ...and 3 more sections

Figures (8)

  • Figure 1: System overview, showing main threads; highlighted (blue) are related to vertex features processing, highlighted (orange) are executed in application main thread, arrows represent event triggers
  • Figure 2: A screenshot live VR while running Mesh2SLAM.
  • Figure 3: Direct geometric projection of vertices to the camera view in Mesh2SLAM.
  • Figure 4: Left: A display of image-based features. Right: Our approach vertex features.
  • Figure 5: View Space, used as reference for vertex feature extraction (obtained from Khronos:2019).
  • ...and 3 more figures