Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Large Language Model with Graph Convolution for Recommendation

Yingpeng Du, Ziyan Wang, Zhu Sun, Haoyan Chua, Hongzhi Liu, Zhonghai Wu, Yining Ma, Jie Zhang, Youchen Sun

TL;DR

This work tackles the mismatch between text-centric LLM reasoning and structured user-item graphs in recommender systems. It proposes GaCLLM, a graph-aware convolutional LLM that uses the LLM as a progressive graph aggregator to enrich user/item descriptions in a least-to-most manner, reducing context-length demands. Through supervised fine-tuning, iterative LLM-based description enhancement, and alignment with GCN embeddings, GaCLLM achieves consistent improvements over state-of-the-art baselines on three real-world datasets, with ablations validating each component. The approach advances practical, scalable recommendation by combining rich textual knowledge with graph structure, enabling more accurate profiling and matching in both job and social recommendation scenarios.

Abstract

In recent years, efforts have been made to use text information for better user profiling and item characterization in recommendations. However, text information can sometimes be of low quality, hindering its effectiveness for real-world applications. With knowledge and reasoning capabilities capsuled in Large Language Models (LLMs), utilizing LLMs emerges as a promising way for description improvement. However, existing ways of prompting LLMs with raw texts ignore structured knowledge of user-item interactions, which may lead to hallucination problems like inconsistent description generation. To this end, we propose a Graph-aware Convolutional LLM method to elicit LLMs to capture high-order relations in the user-item graph. To adapt text-based LLMs with structured graphs, We use the LLM as an aggregator in graph processing, allowing it to understand graph-based information step by step. Specifically, the LLM is required for description enhancement by exploring multi-hop neighbors layer by layer, thereby propagating information progressively in the graph. To enable LLMs to capture large-scale graph information, we break down the description task into smaller parts, which drastically reduces the context length of the token input with each step. Extensive experiments on three real-world datasets show that our method consistently outperforms state-of-the-art methods.

Large Language Model with Graph Convolution for Recommendation

TL;DR

This work tackles the mismatch between text-centric LLM reasoning and structured user-item graphs in recommender systems. It proposes GaCLLM, a graph-aware convolutional LLM that uses the LLM as a progressive graph aggregator to enrich user/item descriptions in a least-to-most manner, reducing context-length demands. Through supervised fine-tuning, iterative LLM-based description enhancement, and alignment with GCN embeddings, GaCLLM achieves consistent improvements over state-of-the-art baselines on three real-world datasets, with ablations validating each component. The approach advances practical, scalable recommendation by combining rich textual knowledge with graph structure, enabling more accurate profiling and matching in both job and social recommendation scenarios.

Abstract

In recent years, efforts have been made to use text information for better user profiling and item characterization in recommendations. However, text information can sometimes be of low quality, hindering its effectiveness for real-world applications. With knowledge and reasoning capabilities capsuled in Large Language Models (LLMs), utilizing LLMs emerges as a promising way for description improvement. However, existing ways of prompting LLMs with raw texts ignore structured knowledge of user-item interactions, which may lead to hallucination problems like inconsistent description generation. To this end, we propose a Graph-aware Convolutional LLM method to elicit LLMs to capture high-order relations in the user-item graph. To adapt text-based LLMs with structured graphs, We use the LLM as an aggregator in graph processing, allowing it to understand graph-based information step by step. Specifically, the LLM is required for description enhancement by exploring multi-hop neighbors layer by layer, thereby propagating information progressively in the graph. To enable LLMs to capture large-scale graph information, we break down the description task into smaller parts, which drastically reduces the context length of the token input with each step. Extensive experiments on three real-world datasets show that our method consistently outperforms state-of-the-art methods.
Paper Structure (34 sections, 12 equations, 7 figures, 4 tables)

This paper contains 34 sections, 12 equations, 7 figures, 4 tables.

Table of Contents

  1. Introduction
  2. RELATED WORK
  3. Graph-based Recommendation
  4. Large Language Models for Recommendation
  5. Large Language Models in Graphs
  6. Preliminaries
  7. Problem Definition
  8. GCN Brief
  9. The proposed method
  10. Supervised Fine-tuning for LLMs
  11. LLM-based Convolutional Inference in Graph
  12. Graph construction.
  13. LLM-based convolutional inference strategy.
  14. Strengths of the LLM-based convolutional strategy.
  15. Aligning GCN-based Embeddings for Recommendation
  16. ...and 19 more sections

Figures (7)

  • Figure 1: (A) The ChatGPT-driven data analysis. (B) Performance of recommendation methods on user subgroups with different description quality levels.
  • Figure 2: Comparison between the token capacity of LLMs and the expected size (left part); Elicit the reasoning capacity of LLMs on the graph in a least-to-most manner (right part).
  • Figure 3: The architecture of the proposed method GaCLLM.
  • Figure 4: Different ways for token usage in LLMs.
  • Figure 5:
  • ...and 2 more figures