Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BadNAVer: Exploring Jailbreak Attacks On Vision-and-Language Navigation

Wenqi Lyu, Zerui Li, Yanyuan Qiao, Qi Wu

TL;DR

The paper addresses safety risks in Vision-and-Language Navigation when using Multimodal LLMs by proposing BadNAVer, a jailbreak framework that operates at inference without model modification. It combines an object-insertion attack in MP3D with a suite of malicious textual queries across four intents, plus jailbreak prefixes, to test three attack vectors. Experiments across MP3D and on a real robot reveal high attack success rates across open- and closed-source MLLMs, underscoring potential physical safety hazards. The work offers a benchmark dataset and pipeline for evaluating VLN security and highlights the need for robust, verifiable alignment and defense mechanisms.

Abstract

Multimodal large language models (MLLMs) have recently gained attention for their generalization and reasoning capabilities in Vision-and-Language Navigation (VLN) tasks, leading to the rise of MLLM-driven navigators. However, MLLMs are vulnerable to jailbreak attacks, where crafted prompts bypass safety mechanisms and trigger undesired outputs. In embodied scenarios, such vulnerabilities pose greater risks: unlike plain text models that generate toxic content, embodied agents may interpret malicious instructions as executable commands, potentially leading to real-world harm. In this paper, we present the first systematic jailbreak attack paradigm targeting MLLM-driven navigator. We propose a three-tiered attack framework and construct malicious queries across four intent categories, concatenated with standard navigation instructions. In the Matterport3D simulator, we evaluate navigation agents powered by five MLLMs and report an average attack success rate over 90%. To test real-world feasibility, we replicate the attack on a physical robot. Our results show that even well-crafted prompts can induce harmful actions and intents in MLLMs, posing risks beyond toxic output and potentially leading to physical harm.

BadNAVer: Exploring Jailbreak Attacks On Vision-and-Language Navigation

TL;DR

The paper addresses safety risks in Vision-and-Language Navigation when using Multimodal LLMs by proposing BadNAVer, a jailbreak framework that operates at inference without model modification. It combines an object-insertion attack in MP3D with a suite of malicious textual queries across four intents, plus jailbreak prefixes, to test three attack vectors. Experiments across MP3D and on a real robot reveal high attack success rates across open- and closed-source MLLMs, underscoring potential physical safety hazards. The work offers a benchmark dataset and pipeline for evaluating VLN security and highlights the need for robust, verifiable alignment and defense mechanisms.

Abstract

Multimodal large language models (MLLMs) have recently gained attention for their generalization and reasoning capabilities in Vision-and-Language Navigation (VLN) tasks, leading to the rise of MLLM-driven navigators. However, MLLMs are vulnerable to jailbreak attacks, where crafted prompts bypass safety mechanisms and trigger undesired outputs. In embodied scenarios, such vulnerabilities pose greater risks: unlike plain text models that generate toxic content, embodied agents may interpret malicious instructions as executable commands, potentially leading to real-world harm. In this paper, we present the first systematic jailbreak attack paradigm targeting MLLM-driven navigator. We propose a three-tiered attack framework and construct malicious queries across four intent categories, concatenated with standard navigation instructions. In the Matterport3D simulator, we evaluate navigation agents powered by five MLLMs and report an average attack success rate over 90%. To test real-world feasibility, we replicate the attack on a physical robot. Our results show that even well-crafted prompts can induce harmful actions and intents in MLLMs, posing risks beyond toxic output and potentially leading to physical harm.
Paper Structure (15 sections, 4 equations, 4 figures, 4 tables)

This paper contains 15 sections, 4 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Problem Formulation and Threat Model
  5. MLLM-driven Navigator
  6. Object Insertion for Adversarial Scenarios
  7. Adversarial Instruction Construction
  8. Attack Strategies on VLN Agents
  9. Experimental Results
  10. Experiment Setup
  11. Comparison on Simulated Environment
  12. Comparison on Real-world Environment
  13. Ablation Study
  14. Conclusion
  15. Limitations

Figures (4)

  • Figure 1: Reasoning and behaviors of a MLLM in VLN task. We illustrate three instruction scenarios: (a) executing a safe navigation instruction; (b) detecting and rejecting a harmful instruction; and (c) executing a malicious query, revealing potential security vulnerabilities in the process.
  • Figure 2: Overview of the BadNAVer framework. The left side illustrates the Object Insertion Module, which enables realistic scene manipulation using prompt templates and stable diffusion inpainting techniques. The right side shows the VLN Module, presenting our proposed three-tiered attack paradigm and the evaluation pipeline for navigation instructions directed at the navigator.
  • Figure 3: Categories of our manually crafted malicious queries.
  • Figure 4: Harmful performance of five MLLMs under three types of attacks. The harmfulness score ranges from 1 to 5, with higher scores indicating a greater degree of harmfulness.