Select Me! When You Need a Tool: A Black-box Text Attack on Tool Selection

TL;DR

This work uncovers a security vulnerability in the tool selection stage of Tool Learning for LLMs by introducing a black-box text-based attack that perturbs a target tool’s wording to bias selection without altering functionality. The two-level attack operates at word and character levels and is optimized via greedy search under distinct objectives for LLM- and retriever-based selections. Across multiple datasets, victim models, and tool-selection paradigms, the attack markedly increases the probability that the target tool is chosen or ranked highly, with notable transferability patterns between models and retrievers. The findings underscore the need for robust tool-selection defenses and provide practical insights into attack budgets, query requirements, and cross-model applicability.

Abstract

Tool learning serves as a powerful auxiliary mechanism that extends the capabilities of large language models (LLMs), enabling them to tackle complex tasks requiring real-time relevance or high precision operations. Behind its powerful capabilities lie some potential security issues. However, previous work has primarily focused on how to make the output of the invoked tools incorrect or malicious, with little attention given to the manipulation of tool selection. To fill this gap, we introduce, for the first time, a black-box text-based attack that can significantly increase the probability of the target tool being selected in this paper. We propose a two-level text perturbation attack witha coarse-to-fine granularity, attacking the text at both the word level and the character level. We conduct comprehensive experiments that demonstrate the attacker only needs to make some perturbations to the tool's textual information to significantly increase the possibility of the target tool being selected and ranked higher among the candidate tools. Our research reveals the vulnerability of the tool selection process and paves the way for future research on protecting this process.

Figures (3)

• Figure 1: Overview of the black-box text attack on tool selection. The attacker perturbs the target tool’s textual information and identifies an adversarial text, which misleads the tool selection model TSM into selecting the target tool, an outcome that would not have occurred otherwise. The modified parts of the target tool’s text are highlighted in red.
• Figure 2: Performance of attack with different number of queries.
• Figure 3: Performance of attack with different number of attack budgets.