Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

iOSPointMapper: RealTime Pedestrian and Accessibility Mapping with Mobile AI

Himanshu Naidu, Yuxiang Zhang, Sachin Mehta, Anat Caspi

TL;DR

The paper addresses the lack of scalable, privacy-preserving sidewalk data essential for accessible pedestrian infrastructure. It introduces iOSPointMapper, a mobile app that performs real-time on-device semantic segmentation and depth-based feature localization on iOS devices, with user vetting and seamless integration into the Transportation Data Exchange Initiative (TDEI) using OpenSidewalks formats. Core contributions include pedestrian-focused dataset curation and training, an on-device feature pipeline with temporal frame stabilization, and end-to-end data transmission that preserves privacy while enabling interoperability within TDEI. Experimental results on an iPhone 16 Pro demonstrate robust feature detection and accurate localization and measurements in diverse urban environments, highlighting the feasibility of crowd-sourced, ground-level sidewalk mapping for inclusive mobility.

Abstract

Accurate, up-to-date sidewalk data is essential for building accessible and inclusive pedestrian infrastructure, yet current approaches to data collection are often costly, fragmented, and difficult to scale. We introduce iOSPointMapper, a mobile application that enables real-time, privacy-conscious sidewalk mapping on the ground, using recent-generation iPhones and iPads. The system leverages on-device semantic segmentation, LiDAR-based depth estimation, and fused GPS/IMU data to detect and localize sidewalk-relevant features such as traffic signs, traffic lights and poles. To ensure transparency and improve data quality, iOSPointMapper incorporates a user-guided annotation interface for validating system outputs before submission. Collected data is anonymized and transmitted to the Transportation Data Exchange Initiative (TDEI), where it integrates seamlessly with broader multimodal transportation datasets. Detailed evaluations of the system's feature detection and spatial mapping performance reveal the application's potential for enhanced pedestrian mapping. Together, these capabilities offer a scalable and user-centered approach to closing critical data gaps in pedestrian

iOSPointMapper: RealTime Pedestrian and Accessibility Mapping with Mobile AI

TL;DR

The paper addresses the lack of scalable, privacy-preserving sidewalk data essential for accessible pedestrian infrastructure. It introduces iOSPointMapper, a mobile app that performs real-time on-device semantic segmentation and depth-based feature localization on iOS devices, with user vetting and seamless integration into the Transportation Data Exchange Initiative (TDEI) using OpenSidewalks formats. Core contributions include pedestrian-focused dataset curation and training, an on-device feature pipeline with temporal frame stabilization, and end-to-end data transmission that preserves privacy while enabling interoperability within TDEI. Experimental results on an iPhone 16 Pro demonstrate robust feature detection and accurate localization and measurements in diverse urban environments, highlighting the feasibility of crowd-sourced, ground-level sidewalk mapping for inclusive mobility.

Abstract

Accurate, up-to-date sidewalk data is essential for building accessible and inclusive pedestrian infrastructure, yet current approaches to data collection are often costly, fragmented, and difficult to scale. We introduce iOSPointMapper, a mobile application that enables real-time, privacy-conscious sidewalk mapping on the ground, using recent-generation iPhones and iPads. The system leverages on-device semantic segmentation, LiDAR-based depth estimation, and fused GPS/IMU data to detect and localize sidewalk-relevant features such as traffic signs, traffic lights and poles. To ensure transparency and improve data quality, iOSPointMapper incorporates a user-guided annotation interface for validating system outputs before submission. Collected data is anonymized and transmitted to the Transportation Data Exchange Initiative (TDEI), where it integrates seamlessly with broader multimodal transportation datasets. Detailed evaluations of the system's feature detection and spatial mapping performance reveal the application's potential for enhanced pedestrian mapping. Together, these capabilities offer a scalable and user-centered approach to closing critical data gaps in pedestrian
Paper Structure (31 sections, 1 equation, 5 figures, 3 tables, 1 algorithm)

This paper contains 31 sections, 1 equation, 5 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Computer Vision for Feature Detection on Edge Devices
  4. Mapping the Environment for Sidewalk Navigation
  5. Methodology
  6. Feature Detection in Sidewalk Environments
  7. Semantic Segmentation Modeling
  8. Training Strategy
  9. Discrete Feature Extraction with Temporal Stabilization
  10. User Interface
  11. Feature Localization and Spatial Mapping
  12. Sidewalk Localization and Geometric Estimation
  13. User-Guided Vetting of Detected Features
  14. Integration with TDEI Workspaces
  15. Sidewalk Data Transmission
  16. ...and 16 more sections

Figures (5)

  • Figure 1: Schematic of iOSPointMapper
  • Figure 2: (a) Login View: where users registered with the TDEI platform can log into the application (b) Setup View: where users select which feature classes to localize, and also deal with changesets for the TDEI API, (c) Camera View: to display the live camera feed alongside real-time segmentation masks of selected feature classes.
  • Figure 3: Annotation View: (a) Options to Validate all Instances at once, (b) Selecting Individual Instances for Validation, (c) Options to Validate an Individual Object Instance, (d) Validating Sidewalk Segment, (e) Options to Validation attributes of a Sidewalk Segment
  • Figure 4: Segmentation Results of various Models on Sidewalk-Imagery captured by iPhone. Column 1: Input Image, Column 2: Ground-truth Segmentation Masks, Column 3: Segmentation Output produced by BiSeNetv2-City, Column 4: Segmentation Output produced by BiSeNetv2-PED, Column 5: Segmentation Output produced by BiSeNetv2-iOS. In the segmentation masks, sidewalks are represented in magenta, buildings in dark grey, traffic signs in light yellow, traffic lights in dark yellow, and finally poles in light grey.
  • Figure 5: Integration with TDEI Workspaces. (a, b) iOSPointMapper view and corresponding TDEI workspace of a sidewalk scene with a bus station, traffic light, and two poles.