Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OpenFLAME: Federated Visual Positioning System to Enable Large-Scale Augmented Reality Applications

Sagar Bharadwaj, Harrison Williams, Luke Wang, Michael Liang, Tao Jin, Srinivasan Seshan, Anthony Rowe

TL;DR

OpenFLAME tackles the privacy and scalability challenges of world-scale Visual Positioning Systems by federating VPS services across independent organizations. It introduces a complete client/server pipeline, including VPS Discoverer, Place Recognizer, semantic masking, robust pose estimation, a privacy-aware Pose Confidence metric, and a VPS Selector plus a Visual features-free Stitcher to fuse poses across service boundaries without sharing raw data. The system is evaluated on 30 indoor locations, showing strong performance in dynamic environments, coherent stitching after limited observations, and effective service selection, with a clear privacy-preserving direction for future work. This federated approach enables broader VPS coverage and robustness for large-scale AR applications, with open-source plans to accelerate adoption.

Abstract

World-scale augmented reality (AR) applications need a ubiquitous 6DoF localization backend to anchor content to the real world consistently across devices. Large organizations such as Google and Niantic are 3D scanning outdoor public spaces in order to build their own Visual Positioning Systems (VPS). These centralized VPS solutions fail to meet the needs of many future AR applications -- they do not cover private indoor spaces because of privacy concerns, regulations, and the labor bottleneck of updating and maintaining 3D scans. In this paper, we present OpenFLAME, a federated VPS backend that allows independent organizations to 3D scan and maintain a separate VPS service for their own spaces. This enables access control of indoor 3D scans, distributed maintenance of the VPS backend, and encourages larger coverage. Sharding of VPS services introduces several unique challenges -- coherency of localization results across spaces, quality control of VPS services, selection of the right VPS service for a location, and many others. We introduce the concept of federated image-based localization and provide reference solutions for managing and merging data across maps without sharing private data.

OpenFLAME: Federated Visual Positioning System to Enable Large-Scale Augmented Reality Applications

TL;DR

OpenFLAME tackles the privacy and scalability challenges of world-scale Visual Positioning Systems by federating VPS services across independent organizations. It introduces a complete client/server pipeline, including VPS Discoverer, Place Recognizer, semantic masking, robust pose estimation, a privacy-aware Pose Confidence metric, and a VPS Selector plus a Visual features-free Stitcher to fuse poses across service boundaries without sharing raw data. The system is evaluated on 30 indoor locations, showing strong performance in dynamic environments, coherent stitching after limited observations, and effective service selection, with a clear privacy-preserving direction for future work. This federated approach enables broader VPS coverage and robustness for large-scale AR applications, with open-source plans to accelerate adoption.

Abstract

World-scale augmented reality (AR) applications need a ubiquitous 6DoF localization backend to anchor content to the real world consistently across devices. Large organizations such as Google and Niantic are 3D scanning outdoor public spaces in order to build their own Visual Positioning Systems (VPS). These centralized VPS solutions fail to meet the needs of many future AR applications -- they do not cover private indoor spaces because of privacy concerns, regulations, and the labor bottleneck of updating and maintaining 3D scans. In this paper, we present OpenFLAME, a federated VPS backend that allows independent organizations to 3D scan and maintain a separate VPS service for their own spaces. This enables access control of indoor 3D scans, distributed maintenance of the VPS backend, and encourages larger coverage. Sharding of VPS services introduces several unique challenges -- coherency of localization results across spaces, quality control of VPS services, selection of the right VPS service for a location, and many others. We introduce the concept of federated image-based localization and provide reference solutions for managing and merging data across maps without sharing private data.

Paper Structure

This paper contains 36 sections, 5 equations, 20 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Example Workflow
  3. Background
  4. Visual positioning
  5. VPS services with ubiquitous coverage
  6. Characteristics of individual VPS services
  7. Localization pipeline
  8. VPS Discoverer
  9. Capture and send images
  10. Place Recognizer
  11. Semantic image masking
  12. Pose estimation
  13. Pose Confidence Calculator
  14. VPS Selector
  15. Visual features-free dynamic VPS Stitcher
  16. ...and 21 more sections

Figures (20)

  • Figure 1: OpenFLAME integrates independent VPS services.
  • Figure 2: Localization pipeline on OpenFLAME.
  • Figure 3: Semantic image masking is used to ignore features from typically dynamic objects, making localization more robust to changes in the environment.
  • Figure 4: The Pose Confidence component estimates how reliable the calculated pose is by comparing the query image with one rendered from that pose.
  • Figure 5: VPS Selector chooses the best VPS service by comparing different VPS trajectory results against on-device VIO tracking.
  • ...and 15 more figures