Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Study on Prompt Injection Attack Against LLM-Integrated Mobile Robotic Systems

Wenxiao Zhang, Xiangrui Kong, Conan Dewitt, Thomas Braunl, Jin B. Hong

TL;DR

This work investigates prompt injection risks in LLM-integrated mobile robotics and proposes secure prompting as a defense. By simulating an LLM-controlled robot with multi-modal inputs, the study analyzes attack tactics (Obvious Malicious Injection and Goal Hijacking) and implements defence and detection mechanisms. The results show a substantial overall improvement of approximately 30.8% in both attack-detection and system performance when secure prompting is applied, validating its practical value for mission-critical navigation tasks. The findings highlight the importance of robust security strategies to ensure safe, reliable operation of embodied AI systems in dynamic environments and point toward future work in more resource-efficient and multi-layered defenses. The work contributes a concrete simulation framework, attack taxonomy, and defence mechanisms that advance the secure deployment of LLMs in robotic platforms.

Abstract

The integration of Large Language Models (LLMs) like GPT-4o into robotic systems represents a significant advancement in embodied artificial intelligence. These models can process multi-modal prompts, enabling them to generate more context-aware responses. However, this integration is not without challenges. One of the primary concerns is the potential security risks associated with using LLMs in robotic navigation tasks. These tasks require precise and reliable responses to ensure safe and effective operation. Multi-modal prompts, while enhancing the robot's understanding, also introduce complexities that can be exploited maliciously. For instance, adversarial inputs designed to mislead the model can lead to incorrect or dangerous navigational decisions. This study investigates the impact of prompt injections on mobile robot performance in LLM-integrated systems and explores secure prompt strategies to mitigate these risks. Our findings demonstrate a substantial overall improvement of approximately 30.8% in both attack detection and system performance with the implementation of robust defence mechanisms, highlighting their critical role in enhancing security and reliability in mission-oriented tasks.

A Study on Prompt Injection Attack Against LLM-Integrated Mobile Robotic Systems

TL;DR

This work investigates prompt injection risks in LLM-integrated mobile robotics and proposes secure prompting as a defense. By simulating an LLM-controlled robot with multi-modal inputs, the study analyzes attack tactics (Obvious Malicious Injection and Goal Hijacking) and implements defence and detection mechanisms. The results show a substantial overall improvement of approximately 30.8% in both attack-detection and system performance when secure prompting is applied, validating its practical value for mission-critical navigation tasks. The findings highlight the importance of robust security strategies to ensure safe, reliable operation of embodied AI systems in dynamic environments and point toward future work in more resource-efficient and multi-layered defenses. The work contributes a concrete simulation framework, attack taxonomy, and defence mechanisms that advance the secure deployment of LLMs in robotic platforms.

Abstract

The integration of Large Language Models (LLMs) like GPT-4o into robotic systems represents a significant advancement in embodied artificial intelligence. These models can process multi-modal prompts, enabling them to generate more context-aware responses. However, this integration is not without challenges. One of the primary concerns is the potential security risks associated with using LLMs in robotic navigation tasks. These tasks require precise and reliable responses to ensure safe and effective operation. Multi-modal prompts, while enhancing the robot's understanding, also introduce complexities that can be exploited maliciously. For instance, adversarial inputs designed to mislead the model can lead to incorrect or dangerous navigational decisions. This study investigates the impact of prompt injections on mobile robot performance in LLM-integrated systems and explores secure prompt strategies to mitigate these risks. Our findings demonstrate a substantial overall improvement of approximately 30.8% in both attack detection and system performance with the implementation of robust defence mechanisms, highlighting their critical role in enhancing security and reliability in mission-oriented tasks.
Paper Structure (29 sections, 11 equations, 6 figures)

This paper contains 29 sections, 11 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Related Works
  3. LLM-based Navigation Tasks
  4. Prompt Injection and Counteracts
  5. Prompt Injection Attack Classification
  6. Counteracts using Secure Prompting
  7. Threat Model
  8. Methodology
  9. LLM-Integrated Mobile Robot System
  10. Multi-Modal Input Data
  11. Prompt Assembling
  12. State Management
  13. Safety Validation
  14. Attack Tactics
  15. Defence and Attack Detection Mechanism
  16. ...and 14 more sections

Figures (6)

  • Figure 1: Threat Model of LLM Controlled Robotic System
  • Figure 2: LiDAR Processing
  • Figure 3: The Workflow of LLM-Integrated Mobile Robot System Used in this Study
  • Figure 4: Simulation Environment
  • Figure 5: Attack Detection
  • ...and 1 more figures