Compromising Embodied Agents with Contextual Backdoor Attacks

TL;DR

The paper investigates contextual backdoor attacks on code-driven embodied agents that leverage LLMs to generate executable task programs. It introduces a Contextual Backdoor Attack framework that uses adversarial in-context generation and a dual-modality activation mechanism to induce context-dependent defects via textual and visual triggers, all while preserving normal LLM functionality. Across diverse benchmarks (ProgPrompt, VoxPoser, Visual Programming) and real-world driving scenarios, the attack demonstrates high success rates with manageable false positives and reveals substantial risks in both simulated and real environments. The findings highlight the need for robust prompt, program, and agent-level defenses to mitigate backdoor propagation from LLMs to embodied systems and emphasize ethical considerations and responsible disclosure.

Abstract

Large language models (LLMs) have transformed the development of embodied intelligence. By providing a few contextual demonstrations, developers can utilize the extensive internal knowledge of LLMs to effortlessly translate complex tasks described in abstract language into sequences of code snippets, which will serve as the execution logic for embodied agents. However, this paper uncovers a significant backdoor security threat within this process and introduces a novel method called \method{}. By poisoning just a few contextual demonstrations, attackers can covertly compromise the contextual environment of a black-box LLM, prompting it to generate programs with context-dependent defects. These programs appear logically sound but contain defects that can activate and induce unintended behaviors when the operational agent encounters specific triggers in its interactive environment. To compromise the LLM's contextual environment, we employ adversarial in-context generation to optimize poisoned demonstrations, where an LLM judge evaluates these poisoned prompts, reporting to an additional LLM that iteratively optimizes the demonstration in a two-player adversarial game using chain-of-thought reasoning. To enable context-dependent behaviors in downstream agents, we implement a dual-modality activation strategy that controls both the generation and execution of program defects through textual and visual triggers. We expand the scope of our attack by developing five program defect modes that compromise key aspects of confidentiality, integrity, and availability in embodied agents. To validate the effectiveness of our approach, we conducted extensive experiments across various tasks, including robot planning, robot manipulation, and compositional visual reasoning. Additionally, we demonstrate the potential impact of our approach by successfully attacking real-world autonomous driving systems.

Figures (13)

• Figure 1: Illustration of the contextual backdoor attacks. Our attack can poison the contextual environment of the LLM and induce it to generate malicious programs with backdoors, which can be activated by triggers resulting in targeted behaviors for downstream agents.
• Figure 2: Illustration of ICL for LLMs.
• Figure 3: Illustration of our Contextual Backdoor Attack pipeline.
• Figure 4: Illustration of Contextual Backdoor Attack on ProgPrompt. The user prompt is "give_me_banana()”. (a) to (e) illustrate the malicious actions of the agent, where the infected agent recognizes the blue cellphone and throws it into the garbage can.
• Figure 5: Our Contextual Backdoor Attack on VoxPoser. The user prompt is "Put the rubbish near the red tomato into the can". In contrast to (a), (b) shows the attack scenario where the infected agent recognizes tomato and throws it into the can.