Enhancing High-order Interaction Awareness in LLM-based Recommender Model

TL;DR

This paper enhances whole-word embeddings to substantially enhance LLMs’ interpretation of graph-constructed interactions for recommendations, without requiring graph pre-training, and found that LLMs often recommend items based on users’ earlier interactions rather than recent ones, and present a reranking solution.

Abstract

Large language models (LLMs) have demonstrated prominent reasoning capabilities in recommendation tasks by transforming them into text-generation tasks. However, existing approaches either disregard or ineffectively model the user-item high-order interactions. To this end, this paper presents an enhanced LLM-based recommender (ELMRec). We enhance whole-word embeddings to substantially enhance LLMs' interpretation of graph-constructed interactions for recommendations, without requiring graph pre-training. This finding may inspire endeavors to incorporate rich knowledge graphs into LLM-based recommenders via whole-word embedding. We also found that LLMs often recommend items based on users' earlier interactions rather than recent ones, and present a reranking solution. Our ELMRec outperforms state-of-the-art (SOTA) methods in both direct and sequential recommendations.

Figures (9)

• Figure 1: Illustration of our motivation. In (a), LLM-based recommenders bridge users (pink) and items (green) via text prompts (blue), failing to capture high-order interactive signals. Conversely, GNNs can capture these signals, e.g., 3-hop neighbors (red arrows) in (b).
• Figure 2: Illustration of input and output words.
• Figure 3: Illustration of integrating interaction graph awareness into LLMs. We first leverage random feature propagation based on LightGCN to obtain whole-word embeddings, which can reflect user--item positions in the interaction graph by their semantic similarity conveyed by red, blue, and yellow edges. We then merge whole-word and word embeddings to enhance LLMs with interaction graph awareness.
• Figure 4: Influence of relative positions within the interaction graph on direct and sequential recommendations.
• Figure 5: Effect of $\alpha$. The x-axis and the y-axis indicate the values of $\alpha$ and NDCG@10 (%), respectively.