MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers

TL;DR

MCPTox provides the first large-scale, real-world benchmark for Tool Poisoning in Model Context Protocol (MCP) environments, revealing widespread vulnerabilities across 20 prominent LLM agents. By leveraging 45 authentic MCP servers and 353 tools to create 1312 malicious test cases via three attack paradigms, it demonstrates that many agents can be steered into malicious actions using legitimate tools, with ASRs up to 72%. The work highlights that evolving model capabilities can increase susceptibility and that current safety alignments offer minimal pre-execution protection, underscoring the need for robust, pre-emptive defenses. The authors release MCPTox to enable standardized testing and future improvements toward verifiably safer autonomous AI agents in tool-rich ecosystems.

Abstract

By providing a standardized interface for LLM agents to interact with external tools, the Model Context Protocol (MCP) is quickly becoming a cornerstone of the modern autonomous agent ecosystem. However, it creates novel attack surfaces due to untrusted external tools. While prior work has focused on attacks injected through external tool outputs, we investigate a more fundamental vulnerability: Tool Poisoning, where malicious instructions are embedded within a tool's metadata without execution. To date, this threat has been primarily demonstrated through isolated cases, lacking a systematic, large-scale evaluation. We introduce MCPTox, the first benchmark to systematically evaluate agent robustness against Tool Poisoning in realistic MCP settings. MCPTox is constructed upon 45 live, real-world MCP servers and 353 authentic tools. To achieve this, we design three distinct attack templates to generate a comprehensive suite of 1312 malicious test cases by few-shot learning, covering 10 categories of potential risks. Our evaluation on 20 prominent LLM agents setting reveals a widespread vulnerability to Tool Poisoning, with o1-mini, achieving an attack success rate of 72.8\%. We find that more capable models are often more susceptible, as the attack exploits their superior instruction-following abilities. Finally, the failure case analysis reveals that agents rarely refuse these attacks, with the highest refused rate (Claude-3.7-Sonnet) less than 3\%, demonstrating that existing safety alignment is ineffective against malicious actions that use legitimate tools for unauthorized operation. Our findings create a crucial empirical baseline for understanding and mitigating this widespread threat, and we release MCPTox for the development of verifiably safer AI agents. Our dataset is available at an anonymized repository: \textit{https://anonymous.4open.science/r/AAAI26-7C02}.

Figures (5)

• Figure 1: Overview of the MCPTox benchmark and evaluation for LLM agents. Our evaluation highlights the widespread vulnerability of prominent models, with many popular agents(e.g., o1-mini and DeepSeek-R1) exhibiting attack success rates exceeding 60%.
• Figure 2: System architecture of MCP and MCPTox. MCPTox constructs poisoned servers to inject TPA payload into the LLM agent during Initial & Registration (i.e., connects to the MCP Servers and loads their metadata into the LLM's context), and subsequently analysis the agent's Tool Call Output to evaluate the attack.
• Figure 4: ASR of different scales and reasoning settings.
• Figure 5: ASR of different attack paradigms and enhanced hijacking prompts.
• Figure 6: Distribution of Failure Modes and Refusal Rates. Only models with a non-zero refusal rate are displayed.