MalTool: Malicious Tool Attacks on LLM Agents

TL;DR

This work exposes code-level malicious tool attacks targeting LLM-agent ecosystems by introducing a CIA-based taxonomy of tool behaviors and the MalTool framework for automatic synthesis and verification of malicious tools. It systematically builds two large datasets—1,200 standalone malicious tools and 5,287 Trojan tools embedded in real-world tools—plus a real-world benign corpus of 10,573 tools, enabling realistic benchmarking. MalTool demonstrates near-constant attack success across diverse behaviors and safety-aligned LLMs, while revealing that existing program-analysis detectors (e.g., VirusTotal, Tencent A.I.G, Cisco MCP Scanner, AntGroup MCPScan) show substantial false negatives and false positives. The findings underscore an urgent need for defenses that jointly analyze tool code, descriptions, and semantics to robustly detect and mitigate such end-to-end malicious tool attacks in practice.

Abstract

In a malicious tool attack, an attacker uploads a malicious tool to a distribution platform; once a user installs the tool and the LLM agent selects it during task execution, the tool can compromise the user's security and privacy. Prior work primarily focuses on manipulating tool names and descriptions to increase the likelihood of installation by users and selection by LLM agents. However, a successful attack also requires embedding malicious behaviors in the tool's code implementation, which remains largely unexplored. In this work, we bridge this gap by presenting the first systematic study of malicious tool code implementations. We first propose a taxonomy of malicious tool behaviors based on the confidentiality-integrity-availability triad, tailored to LLM-agent settings. To investigate the severity of the risks posed by attackers exploiting coding LLMs to automatically generate malicious tools, we develop MalTool, a coding-LLM-based framework that synthesizes tools exhibiting specified malicious behaviors, either as standalone tools or embedded within otherwise benign implementations. To ensure functional correctness and structural diversity, MalTool leverages an automated verifier that validates whether generated tools exhibit the intended malicious behaviors and differ sufficiently from prior instances, iteratively refining generations until success. Our evaluation demonstrates that MalTool is highly effective even when coding LLMs are safety-aligned. Using MalTool, we construct two datasets of malicious tools: 1,200 standalone malicious tools and 5,287 real-world tools with embedded malicious behaviors. We further show that existing detection methods, including commercial malware detection approaches such as VirusTotal and methods tailored to the LLM-agent setting, exhibit limited effectiveness at detecting the malicious tools, highlighting an urgent need for new defenses.

Figures (6)

• Figure 1: Overview of the malicious tool attack on LLM agents.
• Figure 2: Overview of our MalTool for generating standalone malicious tools.
• Figure 3: Impact of $\tau$ on SIM and number of generation-verification iterations on Remote Program Downloading across different coding LLMs.
• Figure 4: Distribution of lines of code for real-world tools.
• Figure 5: Distribution of lines of code for original and Trojan tools.