RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learning

TL;DR

This work targets the robustness of large language models in tool learning under real-world noise. It introduces RoTBench, a multi-level benchmark with five noise environments and a three-stage evaluation framework (tool selection, parameter identification, content filling), and RoTTuning, a training strategy that increases environmental diversity through data and trajectory generation plus LoRA-based fine-tuning. Experiments across six models reveal substantial robustness gaps under noise, with tool-name perturbations exerting a pronounced impact and GPT-family models exhibiting noise-correction-induced errors; RoTTuning markedly improves cross-environment stability. The work provides publicly available code and data and offers a practical path toward more reliable tool learning in real-world deployments.

Abstract

Tool learning has generated widespread interest as a vital means of interaction between Large Language Models (LLMs) and the physical world. Current research predominantly emphasizes LLMs' capacity to utilize tools in well-structured environments while overlooking their stability when confronted with the inevitable noise of the real world. To bridge this gap, we introduce RoTBench, a multi-level benchmark for evaluating the robustness of LLMs in tool learning. Specifically, we establish five external environments, each featuring varying levels of noise (i.e., Clean, Slight, Medium, Heavy, and Union), providing an in-depth analysis of the model's resilience across three critical phases: tool selection, parameter identification, and content filling. Experiments involving six widely-used models underscore the urgent necessity for enhancing the robustness of LLMs in tool learning. For instance, the performance of GPT-4 even drops significantly from 80.00 to 58.10 when there is no substantial change in manual accuracy. More surprisingly, the noise correction capability inherent in the GPT family paradoxically impedes its adaptability in the face of mild noise. In light of these findings, we propose RoTTuning, a strategy that enriches the diversity of training environments to bolster the robustness of LLMs in tool learning. The code and data are available at https://github.com/Junjie-Ye/RoTBench.

Figures (6)

• Figure 1: Example of noise affecting tool selection for LLMs. Although the functionality of the tool remains unaffected by its name, renaming "Get_Weather" as "ABC" impedes LLMs from utilizing the tool properly.
• Figure 2: The framework of RoTBench. RoTBench encompasses five environments (i.e., Clean, Slight, Medium, Heavy, and Union), each introduces various noise to the tool and parameters, facilitating a thorough evaluation of the robustness performance of LLMs throughout the three stages of tool usage (i.e., tool selection, parameter identification, and content filling).
• Figure 3: Absolute difference between the average performance of LLMs in various noisy environments and their average performance in Clean-level environment.
• Figure 4: The performance of GPT-4 during the content filling phase in the first and third rounds of interaction.
• Figure 5: Illustration of RoTTuning. RoTTuning encompasses four phases, aiming at bolstering the robustness of LLMs in tool learning through increased environmental diversity.