Large Language Model Enhanced Hard Sample Identification for Denoising Recommendation

TL;DR

This work constructs an LLM-based scorer to evaluate the semantic consistency of items with the user preference, which is quantified based on summarized historical user interactions, and proposes an iterative preference update module designed to continuously refine summarized user preference, which may be biased due to false-positive user-item interactions.

Abstract

Implicit feedback, often used to build recommender systems, unavoidably confronts noise due to factors such as misclicks and position bias. Previous studies have attempted to alleviate this by identifying noisy samples based on their diverged patterns, such as higher loss values, and mitigating the noise through sample dropping or reweighting. Despite the progress, we observe existing approaches struggle to distinguish hard samples and noise samples, as they often exhibit similar patterns, thereby limiting their effectiveness in denoising recommendations. To address this challenge, we propose a Large Language Model Enhanced Hard Sample Denoising (LLMHD) framework. Specifically, we construct an LLM-based scorer to evaluate the semantic consistency of items with the user preference, which is quantified based on summarized historical user interactions. The resulting scores are used to assess the hardness of samples for the pointwise or pairwise training objectives. To ensure efficiency, we introduce a variance-based sample pruning strategy to filter potential hard samples before scoring. Besides, we propose an iterative preference update module designed to continuously refine summarized user preference, which may be biased due to false-positive user-item interactions. Extensive experiments on three real-world datasets and four backbone recommenders demonstrate the effectiveness of our approach.

Figures (8)

• Figure 1: Loss values and prediction scores during training LightGCN on Yelp dataset. We observe that hard and noisy samples exhibit similar values in prediction score and loss, making it difficult to differentiate them.
• Figure 2: The overview of the LLMHD framework. LLMHD leverages LLMs to differentiate hard and noisy samples, thereby enhancing the denoising recommender training task. The framework identifies hard samples through three main modules: (1) Variance-based Sample Pruning, (2) LLM-based Sample Scoring, and (3) Iterative Preference Updating.
• Figure 3: Performance comparison of denoise training with random noises in Amazon-books.
• Figure 4: Hyper-parameter analysis in $\text{LLMHD}_{\text{BPR}}$ with LightGCN backbone on the Amazon-books.
• Figure 5: Example of user preference summarization process on Amazon-books dataset.