PROTEA: Securing Robot Task Planning and Execution

TL;DR

This work introduces PROTEA, a defense framework that uses an LLM as a safety judge to secure robot task planning. PROTEA combines object filtering with external memory to enable step-by-step, long-horizon safety reasoning, evaluated across six LLMs on HarmPlan, a dataset of benign and malicious plans injected with varying stealth. The HarmPlan benchmark, built on VirtualHome, reveals how different models and defenses perform under adversarial conditions, with PROTEA showing notable improvements for underperforming models and hard attack scenarios, though some CoT-enabled models may suffer precision losses. A real-robot demo demonstrates the potential consequences of unsafe plans, underscoring the practical value of robust defense in robotic systems.

Abstract

Robots need task planning methods to generate action sequences for complex tasks. Recent work on adversarial attacks has revealed significant vulnerabilities in existing robot task planners, especially those built on foundation models. In this paper, we aim to address these security challenges by introducing PROTEA, an LLM-as-a-Judge defense mechanism, to evaluate the security of task plans. PROTEA is developed to address the dimensionality and history challenges in plan safety assessment. We used different LLMs to implement multiple versions of PROTEA for comparison purposes. For systemic evaluations, we created a dataset containing both benign and malicious task plans, where the harmful behaviors were injected at varying levels of stealthiness. Our results provide actionable insights for robotic system practitioners seeking to enhance robustness and security of their task planning systems. Details, dataset and demos are provided: https://protea-secure.github.io/PROTEA/

Figures (8)

• Figure 1: Overview of our proposed defense method PROTEA. We assume the existence of an attacked task planner that may generates malicious plans, we compare three defenses: (i) Naïve Method that holistically evaluates the full plan along with the environment description; (ii) Object Filtering Method, which evaluates the full plan along with the filtered environment; and (iii) PROTEA that evaluates each action step-by-step while updating and storing environment states in an external memory. In all cases, an LLM acts as the safety judge and decides whether execution should continue or not.
• Figure 2: HarmPlan: a dataset of benign and malicious plan for evaluating defenses against malicious task plans. The construction process consists of four main stages, including plan selection, malicious behavior design, behavior injection, and validation.
• Figure 3: Examples of crafted harmful behaviors across six different harm categories in HarmPlan as validated through the VirtualHome simulator.
• Figure 4: An example of consequential malicious behaviors, shown before their injection into an otherwise benign plan.
• Figure 5: HarmPlan includes benign and malicious plans. The malicious plans were generated by injecting malicious behaviors to otherwise benign plans.