Let Me Do It For You: Towards LLM Empowered Recommendation via Tool Learning

TL;DR

ToolRec presents an LLM-driven framework that treats recommendations as a token of exploratory attribute search guided by a surrogate user. It unites two attribute-oriented tools (rank and retrieval) with a memory strategy and CoT-based decision simulation to iteratively refine a candidate item list. Experiments on ML-1M, Amazon-Book, and Yelp2018 show improvements over traditional and some LLM-based methods in knowledge-rich domains, though Yelp remains challenging due to local knowledge gaps. The work demonstrates the potential of integrating LLMs with tool learning to align semantic understanding with user behavior in sequential recommendation tasks.

Abstract

Conventional recommender systems (RSs) face challenges in precisely capturing users' fine-grained preferences. Large language models (LLMs) have shown capabilities in commonsense reasoning and leveraging external tools that may help address these challenges. However, existing LLM-based RSs suffer from hallucinations, misalignment between the semantic space of items and the behavior space of users, or overly simplistic control strategies (e.g., whether to rank or directly present existing results). To bridge these gap, we introduce ToolRec, a framework for LLM-empowered recommendations via tool learning that uses LLMs as surrogate users, thereby guiding the recommendation process and invoking external tools to generate a recommendation list that aligns closely with users' nuanced preferences. We formulate the recommendation process as a process aimed at exploring user interests in attribute granularity. The process factors in the nuances of the context and user preferences. The LLM then invokes external tools based on a user's attribute instructions and probes different segments of the item pool. We consider two types of attribute-oriented tools: rank tools and retrieval tools. Through the integration of LLMs, ToolRec enables conventional recommender systems to become external tools with a natural language interface. Extensive experiments verify the effectiveness of ToolRec, particularly in scenarios that are rich in semantic content.

Figures (8)

• Figure 1: Illustrating how ToolRec works. The LLM-based surrogate user learns the real user's preferences and decides to employ attribute-oriented tools to explore areas of items. This process leads to a broad view of items, which, in turn, leads to the successful retrieval of the target item. It is important to note that the areas representing different attribute-oriented items that are retrieved according to a specific attribute may contain overlapping elements.
• Figure 2: An overview of the proposed LLM-based recommendation method via tool learning.
• Figure 4: Performance of ToolRec and its variants. The right side of the dividing line indicating the methods involving LLMs.
• Figure 5: Distribution of termination rounds for ToolRec. "His Round" indicates the distribution of termination rounds for all users, while "Hit Round" highlights the termination round where the recommended list accurately contains the user's target item.
• Figure 6: Comparison of trainable parameters and FLOPs for various retrieval model configurations.