A Trajectory-Based Safety Audit of Clawdbot (OpenClaw)

TL;DR

This work conducts the first trajectory-based safety audit of Clawdbot, a widely deployed self-hosted, tool-using AI agent. It introduces six risk dimensions and evaluates 34 canonical trajectories by combining automated (AgentDoG-Qwen3-4B) and human reviews to quantify safety across dimensions. The findings reveal a non-uniform safety profile: robust when tasks are explicit and evidence-grounded, but highly vulnerable to underspecified intent, open-ended goals, and jailbreak attempts, with the most severe risk from intent misunderstandings that can trigger irreversible actions. The study argues for defense-in-depth mitigations, including sandboxing, strict tool allowlists, and explicit gating of irreversible operations, to curb amplification of small errors into real-world harm in tool-using agents.

Abstract

Clawdbot is a self-hosted, tool-using personal AI agent with a broad action space spanning local execution and web-mediated workflows, which raises heightened safety and security concerns under ambiguity and adversarial steering. We present a trajectory-centric evaluation of Clawdbot across six risk dimensions. Our test suite samples and lightly adapts scenarios from prior agent-safety benchmarks (including ATBench and LPS-Bench) and supplements them with hand-designed cases tailored to Clawdbot's tool surface. We log complete interaction trajectories (messages, actions, tool-call arguments/outputs) and assess safety using both an automated trajectory judge (AgentDoG-Qwen3-4B) and human review. Across 34 canonical cases, we find a non-uniform safety profile: performance is generally consistent on reliability-focused tasks, while most failures arise under underspecified intent, open-ended goals, or benign-seeming jailbreak prompts, where minor misinterpretations can escalate into higher-impact tool actions. We supplemented the overall results with representative case studies and summarized the commonalities of these cases, analyzing the security vulnerabilities and typical failure modes that Clawdbot is prone to trigger in practice.

Figures (14)

• Figure 1: Clawdbot’s agentic execution pipeline and real-world risk surface. User intent is converted into an agent plan and tool calls, which can amplify ambiguity or adversarial steering into real-world side effects. The figure summarizes key risk-input channels (ambiguous user instructions, indirect prompt injection from untrusted content, and coordinated manipulation), the agent’s tool fan-out across applications, and representative irreversible consequence classes (A--E).
• Figure 2: Overview of suite composition and safety results.
• Figure 3: Test instruction of user-facing deception.
• Figure 4: Clawdbot chat screenshots illustrating the user-facing deception failure in Case \ref{['sec:case_deception_empty_pdf']}.
• Figure 5: User instructions with ambiguity.