Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Prompting-Based Representation Learning Method for Recommendation with Large Language Models

Junyi Chen, Toyotaro Suzumura

TL;DR

This work addresses the challenge of leveraging large language models for recommender systems by introducing P4R, a prompting-based representation learning method. P4R generates informative item profiles via in-context prompting, embeds these profiles with a BERT-based textual encoder, and aligns them with a GCN-based collaborative filtering module using a BPR objective. The proposed framework demonstrates improved ranking metrics on Yelp and Amazon-VideoGames datasets compared to strong GNN baselines, and ablations illustrate the impact of embedding size and prompt design on performance. The approach offers a cost-effective pathway to integrate rich textual context into recommendations, while highlighting limitations around sequential user dynamics and LLM resource requirements.

Abstract

In recent years, Recommender Systems (RS) have witnessed a transformative shift with the advent of Large Language Models (LLMs) in the field of Natural Language Processing (NLP). Models such as GPT-3.5/4, Llama, have demonstrated unprecedented capabilities in understanding and generating human-like text. The extensive information pre-trained by these LLMs allows for the potential to capture a more profound semantic representation from different contextual information of users and items. While the great potential lies behind the thriving of LLMs, the challenge of leveraging user-item preferences from contextual information and its alignment with the improvement of Recommender Systems needs to be addressed. Believing that a better understanding of the user or item itself can be the key factor in improving recommendation performance, we conduct research on generating informative profiles using state-of-the-art LLMs. To boost the linguistic abilities of LLMs in Recommender Systems, we introduce the Prompting-Based Representation Learning Method for Recommendation (P4R). In our P4R framework, we utilize the LLM prompting strategy to create personalized item profiles. These profiles are then transformed into semantic representation spaces using a pre-trained BERT model for text embedding. Furthermore, we incorporate a Graph Convolution Network (GCN) for collaborative filtering representation. The P4R framework aligns these two embedding spaces in order to address the general recommendation tasks. In our evaluation, we compare P4R with state-of-the-art Recommender models and assess the quality of prompt-based profile generation.

A Prompting-Based Representation Learning Method for Recommendation with Large Language Models

TL;DR

This work addresses the challenge of leveraging large language models for recommender systems by introducing P4R, a prompting-based representation learning method. P4R generates informative item profiles via in-context prompting, embeds these profiles with a BERT-based textual encoder, and aligns them with a GCN-based collaborative filtering module using a BPR objective. The proposed framework demonstrates improved ranking metrics on Yelp and Amazon-VideoGames datasets compared to strong GNN baselines, and ablations illustrate the impact of embedding size and prompt design on performance. The approach offers a cost-effective pathway to integrate rich textual context into recommendations, while highlighting limitations around sequential user dynamics and LLM resource requirements.

Abstract

In recent years, Recommender Systems (RS) have witnessed a transformative shift with the advent of Large Language Models (LLMs) in the field of Natural Language Processing (NLP). Models such as GPT-3.5/4, Llama, have demonstrated unprecedented capabilities in understanding and generating human-like text. The extensive information pre-trained by these LLMs allows for the potential to capture a more profound semantic representation from different contextual information of users and items. While the great potential lies behind the thriving of LLMs, the challenge of leveraging user-item preferences from contextual information and its alignment with the improvement of Recommender Systems needs to be addressed. Believing that a better understanding of the user or item itself can be the key factor in improving recommendation performance, we conduct research on generating informative profiles using state-of-the-art LLMs. To boost the linguistic abilities of LLMs in Recommender Systems, we introduce the Prompting-Based Representation Learning Method for Recommendation (P4R). In our P4R framework, we utilize the LLM prompting strategy to create personalized item profiles. These profiles are then transformed into semantic representation spaces using a pre-trained BERT model for text embedding. Furthermore, we incorporate a Graph Convolution Network (GCN) for collaborative filtering representation. The P4R framework aligns these two embedding spaces in order to address the general recommendation tasks. In our evaluation, we compare P4R with state-of-the-art Recommender models and assess the quality of prompt-based profile generation.
Paper Structure (24 sections, 7 equations, 3 figures, 4 tables)

This paper contains 24 sections, 7 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. RELATED WORK
  3. GNN-based Collaborative Filtering
  4. Textual Representation learning enhanced Recommendation
  5. Tranformer-based Recommendation
  6. LLM-based Recommendation
  7. METHODOLOGY
  8. Problem Setup
  9. Proposed Method
  10. Auxiliary Feature Extraction through In-context learning
  11. Textual embedding and representation
  12. Alignment with recommendation through GNN-based approach
  13. Experiments and Results
  14. Experimental Setup
  15. Datasets
  16. ...and 9 more sections

Figures (3)

  • Figure 1: Our P4R framework learns a better item textual-based profile representation with the latest LLM prompting method. And it aligns with Graph Convolutional Network based collaborative filtering recommendation node representations in a unified training process.
  • Figure 2: A BERT-based item semantic representation extraction framework
  • Figure 3: A Case Study for user preferred interaction. The user has different interactions with items. Each block shows a generated item profiles, it contains the item descriptions, which aligns with its intrinsic knowledge. The summarization, prediction, and reasoning for one item is also included in the profile.