ROSBag MCP Server: Analyzing Robot Data with LLMs for Agentic Embodied AI Applications

TL;DR

This work introduces ROSbags-MCP, a Model Context Protocol (MCP) server tailored for analyzing ROS and ROS 2 bag data through natural language interfaces. It delivers domain-specific tooling for core data access, trajectory and LiDAR analysis, and visualization, all accessible via a modular MCP framework and benchmarked in MCP Lab across eight LLM/VLM models. The experimental results reveal substantial disparities in tool-calling capabilities among models, with Kimi K2 and Claude Sonnet 4 achieving the highest task completion rates and smaller models showing weaker multi-step reasoning. The open-source implementation aims to make robotic data analysis more accessible to non-experts while preserving precision, and it highlights design factors that influence effective tool use in agentic embodied AI systems.

Abstract

Agentic AI systems and Physical or Embodied AI systems have been two key research verticals at the forefront of Artificial Intelligence and Robotics, with Model Context Protocol (MCP) increasingly becoming a key component and enabler of agentic applications. However, the literature at the intersection of these verticals, i.e., Agentic Embodied AI, remains scarce. This paper introduces an MCP server for analyzing ROS and ROS 2 bags, allowing for analyzing, visualizing and processing robot data with natural language through LLMs and VLMs. We describe specific tooling built with robotics domain knowledge, with our initial release focused on mobile robotics and supporting natively the analysis of trajectories, laser scan data, transforms, or time series data. This is in addition to providing an interface to standard ROS 2 CLI tools ("ros2 bag list" or "ros2 bag info"), as well as the ability to filter bags with a subset of topics or trimmed in time. Coupled with the MCP server, we provide a lightweight UI that allows the benchmarking of the tooling with different LLMs, both proprietary (Anthropic, OpenAI) and open-source (through Groq). Our experimental results include the analysis of tool calling capabilities of eight different state-of-the-art LLM/VLM models, both proprietary and open-source, large and small. Our experiments indicate that there is a large divide in tool calling capabilities, with Kimi K2 and Claude Sonnet 4 demonstrating clearly superior performance. We also conclude that there are multiple factors affecting the success rates, from the tool description schema to the number of arguments, as well as the number of tools available to the models. The code is available with a permissive license at https://github.com/binabik-ai/mcp-rosbags.

Figures (7)

• Figure 1: The overview of the agentic chain of the ROSbag MCP Server. The diagram illustrates the different components, including the available tools in the MCP server, the capability to read various ROSbag formats from the filesystem, and the overall data flow between the MCP client and server. The MCP Host represents the runtime environment that mediates communication between the LLM client (e.g., Claude Desktop or MCP Lab) and the MCP server, managing JSON-RPC connections to the selected LLM provider.
• Figure 2: System architecture of MCP Lab with remote connection support, illustrating the integration of MCP servers and multiple LLM providers.
• Figure 3: Conversation workflow showing the MCP tool interaction within the Claude Desktop App and MCP-Lab interfaces.
• Figure 4: Description of tools in the initial release of the Rosbags MCP server.
• Figure 5: Task completion and tool usage across different models.