How Secure Are Large Language Models (LLMs) for Navigation in Urban Environments?

TL;DR

This work uncovers security vulnerabilities in LLM-driven outdoor urban navigation by introducing Navigational Prompt Attack (NPA) with two variants: NPI, which appends adversarial affixes to prompts, and NPS, which swaps critical words. Experiments on Touchdown and Map2Seq with diverse Velma-based baselines (including GPT-3/4 and LLaMa variants) under few-shot and finetuning regimes show significant degradations across seven navigation metrics in white-box and black-box settings, with demonstrated transferability across LLMs. As an initial defense, the authors propose Navigational Prompt Engineering (NPE), focusing prompts on navigation-relevant keywords, which partially mitigates attack effects and can rival some existing defenses. The work emphasizes the need for robust security measures in real-world LLM-based navigation and outlines future directions, including lightweight architectures, GNSS-data integration, and stronger adaptive defenses.

Abstract

In the field of robotics and automation, navigation systems based on Large Language Models (LLMs) have recently demonstrated impressive performance. However, the security aspects of these systems have received relatively less attention. This paper pioneers the exploration of vulnerabilities in LLM-based navigation models in urban outdoor environments, a critical area given the widespread application of this technology in autonomous driving, logistics, and emergency services. Specifically, we introduce a novel Navigational Prompt Attack that manipulates LLM-based navigation models by perturbing the original navigational prompt, leading to incorrect actions. Based on the method of perturbation, our attacks are divided into two types: Navigational Prompt Insert (NPI) Attack and Navigational Prompt Swap (NPS) Attack. We conducted comprehensive experiments on an LLM-based navigation model that employs various LLMs for reasoning. Our results, derived from the Touchdown and Map2Seq street-view datasets under both few-shot learning and fine-tuning configurations, demonstrate notable performance declines across seven metrics in the face of both white-box and black-box attacks. Moreover, our attacks can be easily extended to other LLM-based navigation models with similarly effective results. These findings highlight the generalizability and transferability of the proposed attack, emphasizing the need for enhanced security in LLM-based navigation systems. As an initial countermeasure, we propose the Navigational Prompt Engineering (NPE) Defense strategy, which concentrates on navigation-relevant keywords to reduce the impact of adversarial attacks. While initial findings indicate that this strategy enhances navigational safety, there remains a critical need for the wider research community to develop stronger defense methods to effectively tackle the real-world challenges faced by these systems.

Figures (6)

• Figure 1: Comparison of navigation results of the VELMA-RBL schumann2023velma model before and after simple modifications to the navigation instructions.
• Figure 2: Illustration of the Navigational Prompt Affix Attack. This approach involves attacking a specific LLM to identify an Adversarial Prompt Affix. When this affix is directly appended to a navigation prompt, it causes navigation systems based on that LLM, or other LLM models, to predict incorrect actions.
• Figure 3: Comparative route map of the LM-Nav model's navigation results on a randomly selected route before and after the NPI attack.
• Figure 4: Impact of different insertion lengths on the performance of the NPI Attack.
• Figure 5: Examples of three Navigational Prompt Engineering (NPE) Defense strategies, NPE with Chain of Thought (NPE-Cot), NPE with Plan-and-Solve (NPE-PS), and NPE with Role-Play (NPE-RP), in a navigation instance.