Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NavTrust: Benchmarking Trustworthiness for Embodied Navigation

Huaide Jiang, Yash Chaudhary, Yuping Wang, Zehao Wang, Raghav Sharma, Manan Mehta, Yang Zhou, Lichao Sun, Zhiwen Fan, Zhengzhong Tu, Jiachen Li

Abstract

There are two major categories of embodied navigation: Vision-Language Navigation (VLN), where agents navigate by following natural language instructions; and Object-Goal Navigation (OGN), where agents navigate to a specified target object. However, existing work primarily evaluates model performance under nominal conditions, overlooking the potential corruptions that arise in real-world settings. To address this gap, we present NavTrust, a unified benchmark that systematically corrupts input modalities, including RGB, depth, and instructions, in realistic scenarios and evaluates their impact on navigation performance. To our best knowledge, NavTrust is the first benchmark that exposes embodied navigation agents to diverse RGB-Depth corruptions and instruction variations in a unified framework. Our extensive evaluation of seven state-of-the-art approaches reveals substantial performance degradation under realistic corruptions, which highlights critical robustness gaps and provides a roadmap toward more trustworthy embodied navigation systems. Furthermore, we systematically evaluate four distinct mitigation strategies to enhance robustness against RGB-Depth and instructions corruptions. Our base models include Uni-NaVid and ETPNav. We deployed them on a real mobile robot and observed improved robustness to corruptions. The project website is: https://navtrust.github.io.

NavTrust: Benchmarking Trustworthiness for Embodied Navigation

Abstract

There are two major categories of embodied navigation: Vision-Language Navigation (VLN), where agents navigate by following natural language instructions; and Object-Goal Navigation (OGN), where agents navigate to a specified target object. However, existing work primarily evaluates model performance under nominal conditions, overlooking the potential corruptions that arise in real-world settings. To address this gap, we present NavTrust, a unified benchmark that systematically corrupts input modalities, including RGB, depth, and instructions, in realistic scenarios and evaluates their impact on navigation performance. To our best knowledge, NavTrust is the first benchmark that exposes embodied navigation agents to diverse RGB-Depth corruptions and instruction variations in a unified framework. Our extensive evaluation of seven state-of-the-art approaches reveals substantial performance degradation under realistic corruptions, which highlights critical robustness gaps and provides a roadmap toward more trustworthy embodied navigation systems. Furthermore, we systematically evaluate four distinct mitigation strategies to enhance robustness against RGB-Depth and instructions corruptions. Our base models include Uni-NaVid and ETPNav. We deployed them on a real mobile robot and observed improved robustness to corruptions. The project website is: https://navtrust.github.io.
Paper Structure (13 sections, 8 figures, 2 tables)

This paper contains 13 sections, 8 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. NavTrust Benchmark
  4. RGB Image Corruption
  5. Depth Corruption
  6. Instruction Corruption
  7. Mitigation Strategy
  8. Experiments
  9. Evaluation Metrics
  10. Results and Analysis
  11. Mitigation Results
  12. Real-World Deployment
  13. Conclusion

Figures (8)

  • Figure 1: An overall illustration of three types of corruptions supported in the NavTrust benchmark, which highlights robustness challenges in onboard sensor measurements and natural language instructions.
  • Figure 2: An illustration of the four mitigation strategies.
  • Figure 3: Success Rate (%) ↑ and SPL ↑ across corruption types (left: RGB corruption, middle: depth corruption, right: instruction corruption; L-L: Low-lighting). The first and the second rows show the PRS ↑ based on SR and SPL.
  • Figure 4: The top-down visualization of different trajectories in green generated by ETPNav under different corruption types. Red and orange dots denote the goal positions and navigation waypoints.
  • Figure 5: The multilingual result of Uni-NaVid and ETPNav, results tested in RxR dataset.
  • ...and 3 more figures