Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NaviNote: Enabling In-situ Spatial Annotation Authoring to Support Exploration and Navigation for Blind and Low Vision People

Ruijia Chen, Yuheng Wu, Charlie Houseago, Filipe Gaspar, Filippo Aleotti, Dorian Gálvez-López, Oliver Johnston, Diego Mazala, Guillermo Garcia-Hernando, Maryam Bandukda, Gabriel Brostow, Jessica Van Brummelen

TL;DR

NaviNote is developed, which combines vision-based high-precision localization with an agentic architecture to enable voice-based annotation authoring and navigation and showed that it significantly improved navigation performance and supported users in understanding and annotating their surroundings.

Abstract

GPS and smartphones enable users to place location-based annotations, capturing rich environmental context. Previous research demonstrates that blind and low vision (BLV) people can use annotations to explore unfamiliar areas. However, current commercial systems allowing BLV users to create annotations have never been evaluated, and current GPS-based systems can deviate several meters. Motivated by high-accuracy visual positioning technology, we first conducted a formative study with 24 BLV participants to envision a more accurate and inclusive annotation system. Surprisingly, many participants viewed the high-accuracy technology not just as an annotation system but also as a tool for precise last-few-meters navigation. Guided by participant feedback, we developed NaviNote, which combines vision-based high-precision localization with an agentic architecture to enable voice-based annotation authoring and navigation. Evaluating NaviNote with 18 BLV participants showed that it significantly improved navigation performance and supported users in understanding and annotating their surroundings. Based on these findings, we discuss design considerations for future accessible annotation authoring systems.

NaviNote: Enabling In-situ Spatial Annotation Authoring to Support Exploration and Navigation for Blind and Low Vision People

TL;DR

NaviNote is developed, which combines vision-based high-precision localization with an agentic architecture to enable voice-based annotation authoring and navigation and showed that it significantly improved navigation performance and supported users in understanding and annotating their surroundings.

Abstract

GPS and smartphones enable users to place location-based annotations, capturing rich environmental context. Previous research demonstrates that blind and low vision (BLV) people can use annotations to explore unfamiliar areas. However, current commercial systems allowing BLV users to create annotations have never been evaluated, and current GPS-based systems can deviate several meters. Motivated by high-accuracy visual positioning technology, we first conducted a formative study with 24 BLV participants to envision a more accurate and inclusive annotation system. Surprisingly, many participants viewed the high-accuracy technology not just as an annotation system but also as a tool for precise last-few-meters navigation. Guided by participant feedback, we developed NaviNote, which combines vision-based high-precision localization with an agentic architecture to enable voice-based annotation authoring and navigation. Evaluating NaviNote with 18 BLV participants showed that it significantly improved navigation performance and supported users in understanding and annotating their surroundings. Based on these findings, we discuss design considerations for future accessible annotation authoring systems.
Paper Structure (46 sections, 3 figures, 2 tables)

This paper contains 46 sections, 3 figures, 2 tables.

Table of Contents

  1. Evaluation (Study 2)
  2. Participants (n2=18)
  3. Apparatus
  4. Procedure
  5. Tutorial
  6. Navigation Task
  7. Free-Form Exploration
  8. Exit Interview
  9. Analysis
  10. Measures & Statistical Analysis
  11. Qualitative Analysis
  12. Evaluation Study Results
  13. Navigation Experiences
  14. NaviNote Improved Navigation Performance
  15. Easier Navigation Perceived with NaviNote
  16. ...and 31 more sections

Figures (3)

  • Figure 5: (A) The layout of the local public square for the evaluation study (gates omitted). There are 20 benches, 26 trash bins, two set of stairs, one ramp, six stone podiums, three flower beds, two table tennis tables, three historical notice boards, and one statue. (B) The two routes for the navigation task. Participants start from the ramp and navigate to the furthest podium on the left for "Emma" (green route), and to a trash bin on the right back corner of the square for "Ben" (red route).
  • Figure 6: Comparison of users’ subjective ratings between NaviNote and the baseline system using UMUX-LITE (Perceived Effectiveness, Ease of Use) and NASA-TLX (Mental Demand, Physical Demand, Self-rated Performance, Frustration)
  • Figure 7: Left: Distribution of NaviNote’s response time per user query, with mean response time to be 10.8 seconds and median response time 8.6 seconds. Most queries were answered within 15 seconds. Right: Response times for individual participants, with variation between participants primarily due to unstable internet connections in the field.