Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Keyword-driven Retrieval-Augmented Large Language Models for Cold-start User Recommendations

Hai-Dang Kieu, Minh Duc Nguyen, Thanh-Son Nguyen, Dung D. Le

TL;DR

The paper tackles cold-start user recommendations by combining keyword-based profiles with a retrieval-augmented LLM re-ranking framework called KALM4REC. It first retrieves candidate items using a keyword-driven heterogeneous graph (MPG) with TF-IRF edge weights, then re-ranks the candidates with LLM prompts that encode user and item keywords and leverage few-shot examples. Experiments on Yelp and TripAdvisor show that MPG-based retrieval combined with LLM re-ranking improves recall and precision, while using keywords reduces token usage and maintains accuracy. The findings demonstrate the practical viability of in-context prompting and keyword-focused representations for enhancing recommender systems in data-sparse scenarios and across domains.

Abstract

Recent advancements in Large Language Models (LLMs) have shown significant potential in enhancing recommender systems. However, addressing the cold-start recommendation problem, where users lack historical data, remains a considerable challenge. In this paper, we introduce KALM4Rec (Keyword-driven Retrieval-Augmented Large Language Models for Cold-start User Recommendations), a novel framework specifically designed to tackle this problem by requiring only a few input keywords from users in a practical scenario of cold-start user restaurant recommendations. KALM4Rec operates in two main stages: candidates retrieval and LLM-based candidates re-ranking. In the first stage, keyword-driven retrieval models are used to identify potential candidates, addressing LLMs' limitations in processing extensive tokens and reducing the risk of generating misleading information. In the second stage, we employ LLMs with various prompting strategies, including zero-shot and few-shot techniques, to re-rank these candidates by integrating multiple examples directly into the LLM prompts. Our evaluation, using a Yelp restaurant dataset with user reviews from three English-speaking cities, shows that our proposed framework significantly improves recommendation quality. Specifically, the integration of in-context instructions with LLMs for re-ranking markedly enhances the performance of the cold-start user recommender system.

Keyword-driven Retrieval-Augmented Large Language Models for Cold-start User Recommendations

TL;DR

The paper tackles cold-start user recommendations by combining keyword-based profiles with a retrieval-augmented LLM re-ranking framework called KALM4REC. It first retrieves candidate items using a keyword-driven heterogeneous graph (MPG) with TF-IRF edge weights, then re-ranks the candidates with LLM prompts that encode user and item keywords and leverage few-shot examples. Experiments on Yelp and TripAdvisor show that MPG-based retrieval combined with LLM re-ranking improves recall and precision, while using keywords reduces token usage and maintains accuracy. The findings demonstrate the practical viability of in-context prompting and keyword-focused representations for enhancing recommender systems in data-sparse scenarios and across domains.

Abstract

Recent advancements in Large Language Models (LLMs) have shown significant potential in enhancing recommender systems. However, addressing the cold-start recommendation problem, where users lack historical data, remains a considerable challenge. In this paper, we introduce KALM4Rec (Keyword-driven Retrieval-Augmented Large Language Models for Cold-start User Recommendations), a novel framework specifically designed to tackle this problem by requiring only a few input keywords from users in a practical scenario of cold-start user restaurant recommendations. KALM4Rec operates in two main stages: candidates retrieval and LLM-based candidates re-ranking. In the first stage, keyword-driven retrieval models are used to identify potential candidates, addressing LLMs' limitations in processing extensive tokens and reducing the risk of generating misleading information. In the second stage, we employ LLMs with various prompting strategies, including zero-shot and few-shot techniques, to re-rank these candidates by integrating multiple examples directly into the LLM prompts. Our evaluation, using a Yelp restaurant dataset with user reviews from three English-speaking cities, shows that our proposed framework significantly improves recommendation quality. Specifically, the integration of in-context instructions with LLMs for re-ranking markedly enhances the performance of the cold-start user recommender system.
Paper Structure (8 sections, 4 equations, 3 figures, 3 tables)

This paper contains 8 sections, 4 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Proposed Framework
  3. Problem Formulation
  4. Candidates Retrieval
  5. LLM-based Candidates Re-ranking
  6. Experimental Settings
  7. Experimental Results
  8. Discussion and Conclusion

Figures (3)

  • Figure 1: KALM4REC begins by extracting noun phrases from user reviews to build profiles. Candidate restaurants are retrieved based on user's selected terms, then forwarded to the Prompt Generation, where user and restaurant details are incorporated. Finally, LLMs leverage their knowledge to capture user expectations and re-rank the candidates effectively.
  • Figure 2: Our prompt template for re-ranking using keyword.
  • Figure 3: Performance of KALM4REC across various aspects using Gemini Pro