Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NoteLLM: A Retrievable Large Language Model for Note Recommendation

Chao Zhang, Shiwei Wu, Haoxin Zhang, Tong Xu, Yan Gao, Yao Hu, Di Wu, Enhong Chen

TL;DR

A novel unified framework called NoteLLM is proposed, which leverages LLMs to address the item-to-item (I2I) note recommendation and uses Note Compression Prompt to compress a note into a single special token, and further learn the potentially related notes' embeddings via a contrastive learning approach.

Abstract

People enjoy sharing "notes" including their experiences within online communities. Therefore, recommending notes aligned with user interests has become a crucial task. Existing online methods only input notes into BERT-based models to generate note embeddings for assessing similarity. However, they may underutilize some important cues, e.g., hashtags or categories, which represent the key concepts of notes. Indeed, learning to generate hashtags/categories can potentially enhance note embeddings, both of which compress key note information into limited content. Besides, Large Language Models (LLMs) have significantly outperformed BERT in understanding natural languages. It is promising to introduce LLMs into note recommendation. In this paper, we propose a novel unified framework called NoteLLM, which leverages LLMs to address the item-to-item (I2I) note recommendation. Specifically, we utilize Note Compression Prompt to compress a note into a single special token, and further learn the potentially related notes' embeddings via a contrastive learning approach. Moreover, we use NoteLLM to summarize the note and generate the hashtag/category automatically through instruction tuning. Extensive validations on real scenarios demonstrate the effectiveness of our proposed method compared with the online baseline and show major improvements in the recommendation system of Xiaohongshu.

NoteLLM: A Retrievable Large Language Model for Note Recommendation

TL;DR

A novel unified framework called NoteLLM is proposed, which leverages LLMs to address the item-to-item (I2I) note recommendation and uses Note Compression Prompt to compress a note into a single special token, and further learn the potentially related notes' embeddings via a contrastive learning approach.

Abstract

People enjoy sharing "notes" including their experiences within online communities. Therefore, recommending notes aligned with user interests has become a crucial task. Existing online methods only input notes into BERT-based models to generate note embeddings for assessing similarity. However, they may underutilize some important cues, e.g., hashtags or categories, which represent the key concepts of notes. Indeed, learning to generate hashtags/categories can potentially enhance note embeddings, both of which compress key note information into limited content. Besides, Large Language Models (LLMs) have significantly outperformed BERT in understanding natural languages. It is promising to introduce LLMs into note recommendation. In this paper, we propose a novel unified framework called NoteLLM, which leverages LLMs to address the item-to-item (I2I) note recommendation. Specifically, we utilize Note Compression Prompt to compress a note into a single special token, and further learn the potentially related notes' embeddings via a contrastive learning approach. Moreover, we use NoteLLM to summarize the note and generate the hashtag/category automatically through instruction tuning. Extensive validations on real scenarios demonstrate the effectiveness of our proposed method compared with the online baseline and show major improvements in the recommendation system of Xiaohongshu.
Paper Structure (21 sections, 4 equations, 2 figures, 6 tables)

This paper contains 21 sections, 4 equations, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. I2I Recommendation
  4. LLMs for Recommendation
  5. Hashtag/Category Generation from Text
  6. Problem Definition
  7. Methodology
  8. Framework of NoteLLM
  9. Note Compression Prompt
  10. Generative-Contrastive Learning
  11. Collaborative Supervised Fine-Tuning
  12. Experiments
  13. Dataset and Experiment Setting
  14. Offline Performance Evaluation
  15. Effect on Different Exposure Notes
  16. ...and 6 more sections

Figures (2)

  • Figure 2: The NoteLLM framework uses a unified prompt for I2I note recommendations and hashtag/category generation. Notes are compressed via the Note Compression Prompt and processed by pre-trained LLMs. We utilize the co-occurrence mechanism to construct the related note pairs and train the I2I recommendation task using Generative-Contrasting Learning. NoteLLM also extracts note's key concepts for hashtag/category generation, enhancing the I2I recommendation task.
  • Figure 3: The visualization cases of NoteLLM and other baselines. Figure 3(a) and 3(b) show the cases in I2I recommendation tasks, where the left query note is the user's clicked note, and the remaining notes are the top-$1$ ranked results retrieved by different methods. Figure 3(c) and 3(d) show the cases in hashtag generation tasks. RedHashtag is the online hashtag generation method. GT means the ground truth hashtags.