PEToolLLM: Towards Personalized Tool Learning in Large Language Models

TL;DR

The work addresses the problem of aligning LLM tool usage with individual user preferences by leveraging interaction history to infer implicit needs. It introduces PEToolBench to simulate history-aware tool usage and PEToolLLaMA, a two-stage framework with SFT and DPO, to learn and optimize personalized tool calls. Empirical results show PEToolLLaMA consistently outperforms both open- and closed-source baselines, achieving substantial gains in tool selection and parameter accuracy across diverse settings. By providing a benchmark, methodology, and extensive analysis, the paper demonstrates the practical potential of personalized tool-enabled LLMs for enhancing user experiences in tool-enabled tasks.

Abstract

Tool learning has emerged as a promising direction by extending Large Language Models' (LLMs) capabilities with external tools. Existing tool learning studies primarily focus on the general-purpose tool-use capability, which addresses explicit user requirements in instructions. However, they overlook the importance of personalized tool-use capability, leading to an inability to handle implicit user preferences. To address the limitation, we first formulate the task of personalized tool learning, which integrates user's interaction history towards personalized tool usage. To fill the gap of missing benchmarks, we construct PEToolBench, featuring diverse user preferences reflected in interaction history under three distinct personalized settings, and encompassing a wide range of tool-use scenarios. Moreover, we propose a framework PEToolLLaMA to adapt LLMs to the personalized tool learning task, which is trained through supervised fine-tuning and direct preference optimization. Extensive experiments on PEToolBench demonstrate the superiority of PEToolLLaMA over existing LLMs.

Figures (13)

• Figure 1: Comparison between (a) tool learning and (b) personalized tool learning. Personalized tool learning facilitates implicit preference comprehension and customized tool usage for individual users.
• Figure 2: Illustration of the process for constructing our PEToolBench.
• Figure 3: Statistics of data instances in three personalized settings (in the left figure) and distributions of interaction history length (in the right figure).
• Figure 4: Distributions of tool categories.
• Figure 5: Performance comparison of tool accuracy when provided with and without interaction history.