Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MobileARLoc: On-device Robust Absolute Localisation for Pervasive Markerless Mobile AR

Changkun Liu, Yukun Zhao, Tristan Braud

TL;DR

MobileARLoc tackles the challenge of achieving accurate, on-device absolute localisation for markerless mobile AR by fusing an absolute pose regressor (APR) with a local VIO tracker. The framework uses VIO to identify reliable APR predictions and employs a two-stage loop (Alignment and Pose Optimization) to align APR and VIO coordinate frames, refine unreliable poses, and compensate VIO drift via a rigid transform. Across outdoor and indoor scenes, MobileARLoc substantially improves translation and rotation accuracy for APRs (up to ~50% translation and ~66% rotation improvements) while maintaining fast on-device inference (around 80 ms) and low storage demands. The approach is APR-agnostic, requires no extra unlabeled data, and demonstrates the feasibility of robust, large-scale, markerless mobile AR on resource-constrained devices. This work therefore enables scalable, privacy-preserving on-device localization for pervasive AR applications.

Abstract

Recent years have seen significant improvement in absolute camera pose estimation, paving the way for pervasive markerless Augmented Reality (AR). However, accurate absolute pose estimation techniques are computation- and storage-heavy, requiring computation offloading. As such, AR systems rely on visual-inertial odometry (VIO) to track the device's relative pose between requests to the server. However, VIO suffers from drift, requiring frequent absolute repositioning. This paper introduces MobileARLoc, a new framework for on-device large-scale markerless mobile AR that combines an absolute pose regressor (APR) with a local VIO tracking system. Absolute pose regressors (APRs) provide fast on-device pose estimation at the cost of reduced accuracy. To address APR accuracy and reduce VIO drift, MobileARLoc creates a feedback loop where VIO pose estimations refine the APR predictions. The VIO system identifies reliable predictions of APR, which are then used to compensate for the VIO drift. We comprehensively evaluate MobileARLoc through dataset simulations. MobileARLoc halves the error compared to the underlying APR and achieve fast (80\,ms) on-device inference speed.

MobileARLoc: On-device Robust Absolute Localisation for Pervasive Markerless Mobile AR

TL;DR

MobileARLoc tackles the challenge of achieving accurate, on-device absolute localisation for markerless mobile AR by fusing an absolute pose regressor (APR) with a local VIO tracker. The framework uses VIO to identify reliable APR predictions and employs a two-stage loop (Alignment and Pose Optimization) to align APR and VIO coordinate frames, refine unreliable poses, and compensate VIO drift via a rigid transform. Across outdoor and indoor scenes, MobileARLoc substantially improves translation and rotation accuracy for APRs (up to ~50% translation and ~66% rotation improvements) while maintaining fast on-device inference (around 80 ms) and low storage demands. The approach is APR-agnostic, requires no extra unlabeled data, and demonstrates the feasibility of robust, large-scale, markerless mobile AR on resource-constrained devices. This work therefore enables scalable, privacy-preserving on-device localization for pervasive AR applications.

Abstract

Recent years have seen significant improvement in absolute camera pose estimation, paving the way for pervasive markerless Augmented Reality (AR). However, accurate absolute pose estimation techniques are computation- and storage-heavy, requiring computation offloading. As such, AR systems rely on visual-inertial odometry (VIO) to track the device's relative pose between requests to the server. However, VIO suffers from drift, requiring frequent absolute repositioning. This paper introduces MobileARLoc, a new framework for on-device large-scale markerless mobile AR that combines an absolute pose regressor (APR) with a local VIO tracking system. Absolute pose regressors (APRs) provide fast on-device pose estimation at the cost of reduced accuracy. To address APR accuracy and reduce VIO drift, MobileARLoc creates a feedback loop where VIO pose estimations refine the APR predictions. The VIO system identifies reliable predictions of APR, which are then used to compensate for the VIO drift. We comprehensively evaluate MobileARLoc through dataset simulations. MobileARLoc halves the error compared to the underlying APR and achieve fast (80\,ms) on-device inference speed.
Paper Structure (19 sections, 10 equations, 3 figures, 3 tables)

This paper contains 19 sections, 10 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Absolute Pose Regression
  4. Uncertainty estimation and Pose Optimization
  5. Proposed Approach
  6. Definition
  7. Detecting reliable pose estimations using VIO
  8. MobileARLoc framework
  9. Dataset
  10. Implementation
  11. Desktop Implementation
  12. Application Implementation
  13. System Performance
  14. Desktop Implementation
  15. Mobile Implementation
  16. ...and 4 more sections

Figures (3)

  • Figure 1: MobileARLoc framework in a mobile AR system.
  • Figure 2: Pose predictions for APR (left) and $\text{APR}^{vio}$ (right) for one test sequence in the Square scene. Using MobileARLoc significantly decreases the number of predictions with large error as well as the noisiness of pose estimation error compared to APR alone.
  • Figure 3: Pose error distribution for Church scene. Using MobileARLoc significantly decreases the number of predictions with large error.