Table of Contents
Fetching ...

Real Faults in Model Context Protocol (MCP) Software: a Comprehensive Taxonomy

Mina Taraghi, Mohammad Mehdi Morovati, Foutse Khomh

TL;DR

The first large-scale taxonomy of faults in MCP servers, comprising five high-level fault categories derived from empirical evidence is presented, providing actionable insights for researchers and practitioners by identifying the most error-prone and critical components of MCP-based systems.

Abstract

The rapid adoption of foundation models has significantly expanded the capabilities of software systems, enabling them to perform complex language, reasoning, and interaction tasks that were previously difficult to automate. However, this progress has also introduced novel challenges that were largely absent in previous generations of software. In particular, the increasing integration of foundation models with external tools and resources raises new concerns regarding reliability, security, and robustness. The Model Context Protocol (MCP) has recently been proposed to standardize interactions between AI-based software systems, software tools, and external resources. Despite its growing adoption, there remains limited systematic understanding of real-world faults in MCP-based software systems. In this paper, we present the first large-scale taxonomy of faults in MCP servers, comprising five high-level fault categories derived from empirical evidence. To evaluate the completeness and generalizability of this taxonomy, we conduct a survey of MCP practitioners with diverse roles and experience levels. The results confirm that all identified fault categories occur in practice and reveal distinct characteristics that differentiate MCP-specific faults from non-MCP faults. Overall, this study provides actionable insights for researchers and practitioners by identifying the most error-prone and critical components of MCP-based systems. These insights can inform the development of more robust, reliable, and secure AI-enabled software systems that rely on MCP.

Real Faults in Model Context Protocol (MCP) Software: a Comprehensive Taxonomy

TL;DR

The first large-scale taxonomy of faults in MCP servers, comprising five high-level fault categories derived from empirical evidence is presented, providing actionable insights for researchers and practitioners by identifying the most error-prone and critical components of MCP-based systems.

Abstract

The rapid adoption of foundation models has significantly expanded the capabilities of software systems, enabling them to perform complex language, reasoning, and interaction tasks that were previously difficult to automate. However, this progress has also introduced novel challenges that were largely absent in previous generations of software. In particular, the increasing integration of foundation models with external tools and resources raises new concerns regarding reliability, security, and robustness. The Model Context Protocol (MCP) has recently been proposed to standardize interactions between AI-based software systems, software tools, and external resources. Despite its growing adoption, there remains limited systematic understanding of real-world faults in MCP-based software systems. In this paper, we present the first large-scale taxonomy of faults in MCP servers, comprising five high-level fault categories derived from empirical evidence. To evaluate the completeness and generalizability of this taxonomy, we conduct a survey of MCP practitioners with diverse roles and experience levels. The results confirm that all identified fault categories occur in practice and reveal distinct characteristics that differentiate MCP-specific faults from non-MCP faults. Overall, this study provides actionable insights for researchers and practitioners by identifying the most error-prone and critical components of MCP-based systems. These insights can inform the development of more robust, reliable, and secure AI-enabled software systems that rely on MCP.
Paper Structure (34 sections, 5 figures, 7 tables)

This paper contains 34 sections, 5 figures, 7 tables.

Figures (5)

  • Figure 1: A high-level view of Model Context Protocol (MCP) Architecture
  • Figure 2: High-level view of the used methodology
  • Figure 3: Taxonomy of MCP server issues (counts in parentheses).
  • Figure 4: Comparison of faults' characteristics for the 5 main categories of bugs in terms of metrics including (a) Required time to fix the fault, (b) Number of comments on the issue, (c) Number of collaborators on issue, (d) Number of comments/collaborators, and (e) Experience level of the developer who fixes the issue
  • Figure 5: Comparison of faults' characteristics for MCP vs. Non-MCP