ToolScope: Enhancing LLM Agent Tool Use through Tool Merging and Context-Aware Filtering

TL;DR

ToolScope addresses two bottlenecks in LLM agent tool use: semantic overlap in tool libraries and the context-length limits that constrain retrieval. It introduces ToolScopeMerger to automatically merge overlapping tools and ToolScopeRetriever to rank and compress toolsets using a dense-sparse hybrid retrieval with reranking. Across three open benchmarks (BFCL, Seal-Tools, UltraTool) and three LLMs, ToolScope improves tool selection accuracy by substantial margins, including up to 38.6% on UltraTool and 34.6% on Seal-Tools. The work demonstrates that automated merging plus context-aware retrieval yields robust gains and highlights remaining benchmark overlap as a limitation.

Abstract

Large language model (LLM) agents rely on external tools to solve complex tasks, but real-world toolsets often contain redundant tools with overlapping names and descriptions, introducing ambiguity and reducing selection accuracy. LLMs also face strict input context limits, preventing efficient consideration of large toolsets. To address these challenges, we propose ToolScope, which includes: (1) ToolScopeMerger with Auto-Correction to automatically audit and fix tool merges, reducing redundancy, and (2) ToolScopeRetriever to rank and select only the most relevant tools for each query, compressing toolsets to fit within context limits without sacrificing accuracy. Evaluations on three state-of-the-art LLMs and three open-source tool-use benchmarks show gains of 8.38% to 38.6% in tool selection accuracy, demonstrating ToolScope's effectiveness in enhancing LLM tool use.

Figures (18)

• Figure 1: A simplified overview of using ToolScope for a tool selection task.
• Figure 2: An overview of ToolScope which consists: (a) ToolScopeMerger, steps including: (1) Candidate Generation, (2) Relationship Classification, (3) Auto-Correction and Merging), (b) ToolScopeRetriever, steps including: (4) Query Decomposition, (5) Retrieval, (6) Reranking and (7) Normalization. Retrieved tools are then presented to LLM Agent for tool selection.
• Figure 3: T-SNE visualization of original BFCL tool embedding and merged BFCL tool embedding
• Figure 4: Sensitivity analysis of ToolScopeMerger on Seal-Tools using different cosine similarity filtering thresholds.
• Figure 5: CSR scores for BFCL, Seal-Tools, and UltraTool by tool quality (Low/Medium/High). $n$ denotes the number of evaluation queries for each quality level used to compute the CSR.