Language-Model Prior Overcomes Cold-Start Items

TL;DR

This paper introduces a novel approach for cold-start item recommendation that utilizes the language model (LM) to estimate item similarities, which are further integrated as a Bayesian prior with classic recommender systems.

Abstract

The growth of recommender systems (RecSys) is driven by digitization and the need for personalized content in areas such as e-commerce and video streaming. The content in these systems often changes rapidly and therefore they constantly face the ongoing cold-start problem, where new items lack interaction data and are hard to value. Existing solutions for the cold-start problem, such as content-based recommenders and hybrid methods, leverage item metadata to determine item similarities. The main challenge with these methods is their reliance on structured and informative metadata to capture detailed item similarities, which may not always be available. This paper introduces a novel approach for cold-start item recommendation that utilizes the language model (LM) to estimate item similarities, which are further integrated as a Bayesian prior with classic recommender systems. This approach is generic and able to boost the performance of various recommenders. Specifically, our experiments integrate it with both sequential and collaborative filtering-based recommender and evaluate it on two real-world datasets, demonstrating the enhanced performance of the proposed approach.

Figures (4)

• Figure 1: Recommender is trained on historical user-item interactions, and then used to recommend new items, including those that previously appeared (i.e., dress) and newly introduced cold-start items (i.e., book).
• Figure 2: The probabilistic model of our proposed method. Overall, $E_i$ is the metadata of item $i$, $X_i$ is the LMs embedding encoded from $E_i$ serving as the prior knowledge when learning $Z_i$, and $Z_i$ is the learned item embedding by the recommender from $E_i$ and $X_i$. To recommend the item at time step $t$, we leverage all items that have interacted with the user $j$ to learn a distribution over all items in ${\mathcal{I}}$, among which we recommend the one with the highest likelihood.
• Figure 3: Tuning hyperparameter $\rho$: overall performance on testing set of SASRec-ours regarding Normalized Discounted Cumulative Gain (NDCG).
• Figure 4: Tuning hyperparameter $\rho$: overall performance on testing set of SASRec-ours regarding Hit Ratio (HR).