Enhancing Collaborative Semantics of Language Model-Driven Recommendations via Graph-Aware Learning

TL;DR

This work tackles the misalignment between LLM semantic spaces and user-item collaborative information in recommender systems. It introduces GAL-Rec, which combines LLM-based reasoning with graph neural network–style multi-hop aggregation and graph-aware contrastive learning, supported by a MoCo-inspired dynamic queue for negative sampling; the overall objective couples cross-entropy with three contrastive terms: $L_{GAL} = L_0 + \lambda_1 L_1 + \lambda_2 L_2 + \lambda_3 L_3 + \lambda_4 \|\Theta\|^2_2$. External embeddings and task-specific prompts seed the LLM with both semantic item content and collaborative signals, while prompt construction mirrors one- and two-hop neighbor aggregation. Empirical results on Amazon Beauty, Toys, and Yelp demonstrate substantial improvements over state-of-the-art baselines, with ablations confirming the value of each component and analysis showing improved multi-hop representation uniformity. The approach offers a scalable, self-supervised pathway to harness graph structure for LLM-driven recommendations, with potential extensions to decoder-only LLMs and multimodal tasks influencing practical recommender systems.

Abstract

Large Language Models (LLMs) are increasingly prominent in the recommendation systems domain. Existing studies usually utilize in-context learning or supervised fine-tuning on task-specific data to align LLMs into recommendations. However, the substantial bias in semantic spaces between language processing tasks and recommendation tasks poses a nonnegligible challenge. Specifically, without the adequate capturing ability of collaborative information, existing modeling paradigms struggle to capture behavior patterns within community groups, leading to LLMs' ineffectiveness in discerning implicit interaction semantic in recommendation scenarios. To address this, we consider enhancing the learning capability of language model-driven recommendation models for structured data, specifically by utilizing interaction graphs rich in collaborative semantics. We propose a Graph-Aware Learning for Language Model-Driven Recommendations (GAL-Rec). GAL-Rec enhances the understanding of user-item collaborative semantics by imitating the intent of Graph Neural Networks (GNNs) to aggregate multi-hop information, thereby fully exploiting the substantial learning capacity of LLMs to independently address the complex graphs in the recommendation system. Sufficient experimental results on three real-world datasets demonstrate that GAL-Rec significantly enhances the comprehension of collaborative semantics, and improves recommendation performance.

Figures (5)

• Figure 1: The differences between LLMs and traditional models in distinguishing items.
• Figure 2: An overall illustration of our proposed learning architecture. The left panel shows the selection of information with different hops, and the right performs graph-aware learning.
• Figure 3: A case demonstrating the motivation of designing task-specific tokens.
• Figure 4: Visualization: Feature distribution of multi-hop information representations of users(items) learn from Toys in two-dimensional space (The darker the color, the more users fall within that area).
• Figure 5: Effect of $\tau$ and $length$ , $neighbors \ counts$ in GAL-Rec.