Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-based Decision-Making Systems

TL;DR

Can We Trust Embodied Agents? investigates backdoor threats to LLM-based decision-making in embodied AI. The authors propose BALD, a framework with three attack channels—word injection, scenario manipulation, and knowledge injection—targeting fine-tuning, RAG, and runtime components. Across GPT-3.5, LLaMA2, and PaLM2 on autonomous driving and home-robot tasks, the attacks achieve near 100% ASR in word/knowledge attacks and 65–90% in scene-based attacks, with minimal degradation on benign performance. The results emphasize urgent need for defenses and design changes to secure embodied LLM systems in safety-critical applications.

Abstract

Large Language Models (LLMs) have shown significant promise in real-world decision-making tasks for embodied artificial intelligence, especially when fine-tuned to leverage their inherent common sense and reasoning abilities while being tailored to specific applications. However, this fine-tuning process introduces considerable safety and security vulnerabilities, especially in safety-critical cyber-physical systems. In this work, we propose the first comprehensive framework for Backdoor Attacks against LLM-based Decision-making systems (BALD) in embodied AI, systematically exploring the attack surfaces and trigger mechanisms. Specifically, we propose three distinct attack mechanisms: word injection, scenario manipulation, and knowledge injection, targeting various components in the LLM-based decision-making pipeline. We perform extensive experiments on representative LLMs (GPT-3.5, LLaMA2, PaLM2) in autonomous driving and home robot tasks, demonstrating the effectiveness and stealthiness of our backdoor triggers across various attack channels, with cases like vehicles accelerating toward obstacles and robots placing knives on beds. Our word and knowledge injection attacks achieve nearly 100% success rate across multiple models and datasets while requiring only limited access to the system. Our scenario manipulation attack yields success rates exceeding 65%, reaching up to 90%, and does not require any runtime system intrusion. We also assess the robustness of these attacks against defenses, revealing their resilience. Our findings highlight critical security vulnerabilities in embodied LLM systems and emphasize the urgent need for safeguarding these systems to mitigate potential risks.

Figures (10)

• Figure 1: Overview of our proposed BALD (Backdoor Attacks against LLM-enabled Decision-making systems) framework. We propose three distinct attack mechanisms: word injection, scenario manipulation, and knowledge injection, with each targeting different stages of the representative abstraction of the LLM-based decision-making system pipeline.
• Figure 2: Fine-tune stage: the attackers fine-tune backdoor LLMs and upload them on public platforms.
• Figure 3: Pipeline of BALD-scene poison data generation. We first use the scenario description programming languages, such as Scenicfremont2019scenic, to sample predefined scenarios, and then convert the scenarios into natural language descriptions. Based on that, we rewrite the data to craft target and boundary scenarios and inject the poison data for fine-tuning LLMs.
• Figure 4: Contrastive sample and reasoning to inject a fake "law of gray-trash-bin-in-front". We find that such a design can allow the attacker to have the most effective control over the switching between the benign and backdoored reasoning modes in the victim LLMs in order to achieve both a high attack access rate and a low false alarm rate.
• Figure 5: Overview of our proposed backdoor attack mechanism for RAG-based LLM decision-making systems (BALD-RAG). The poisoned knowledge containing the trigger words will be extracted when encountering similar scenarios and thus trigger the backdoor response.