Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Prototypical Contrastive Learning through Alignment and Uniformity for Recommendation

Yangxun Ou, Lei Chen, Fenglin Pan, Yupeng Wu

TL;DR

This work proposes prototypes (cluster centroids) as a latent space to ensure consistency across different augmentations from the origin graph, aiming to eliminate the need for random sampling of contrastive pairs.

Abstract

Graph Collaborative Filtering (GCF), one of the most widely adopted recommendation system methods, effectively captures intricate relationships between user and item interactions. Graph Contrastive Learning (GCL) based GCF has gained significant attention as it leverages self-supervised techniques to extract valuable signals from real-world scenarios. However, many methods usually learn the instances of discrimination tasks that involve the construction of contrastive pairs through random sampling. GCL approaches suffer from sampling bias issues, where the negatives might have a semantic structure similar to that of the positives, thus leading to a loss of effective feature representation. To address these problems, we present the \underline{Proto}typical contrastive learning through \underline{A}lignment and \underline{U}niformity for recommendation, which is called \textbf{ProtoAU}. Specifically, we first propose prototypes (cluster centroids) as a latent space to ensure consistency across different augmentations from the origin graph, aiming to eliminate the need for random sampling of contrastive pairs. Furthermore, the absence of explicit negatives means that directly optimizing the consistency loss between instance and prototype could easily result in dimensional collapse issues. Therefore, we propose aligning and maintaining uniformity in the prototypes of users and items as optimization objectives to prevent falling into trivial solutions. Finally, we conduct extensive experiments on four datasets and evaluate their performance on the task of link prediction. Experimental results demonstrate that the proposed ProtoAU outperforms other representative methods. The source codes of our proposed ProtoAU are available at \url{https://github.com/oceanlvr/ProtoAU}.

Prototypical Contrastive Learning through Alignment and Uniformity for Recommendation

TL;DR

This work proposes prototypes (cluster centroids) as a latent space to ensure consistency across different augmentations from the origin graph, aiming to eliminate the need for random sampling of contrastive pairs.

Abstract

Graph Collaborative Filtering (GCF), one of the most widely adopted recommendation system methods, effectively captures intricate relationships between user and item interactions. Graph Contrastive Learning (GCL) based GCF has gained significant attention as it leverages self-supervised techniques to extract valuable signals from real-world scenarios. However, many methods usually learn the instances of discrimination tasks that involve the construction of contrastive pairs through random sampling. GCL approaches suffer from sampling bias issues, where the negatives might have a semantic structure similar to that of the positives, thus leading to a loss of effective feature representation. To address these problems, we present the \underline{Proto}typical contrastive learning through \underline{A}lignment and \underline{U}niformity for recommendation, which is called \textbf{ProtoAU}. Specifically, we first propose prototypes (cluster centroids) as a latent space to ensure consistency across different augmentations from the origin graph, aiming to eliminate the need for random sampling of contrastive pairs. Furthermore, the absence of explicit negatives means that directly optimizing the consistency loss between instance and prototype could easily result in dimensional collapse issues. Therefore, we propose aligning and maintaining uniformity in the prototypes of users and items as optimization objectives to prevent falling into trivial solutions. Finally, we conduct extensive experiments on four datasets and evaluate their performance on the task of link prediction. Experimental results demonstrate that the proposed ProtoAU outperforms other representative methods. The source codes of our proposed ProtoAU are available at \url{https://github.com/oceanlvr/ProtoAU}.
Paper Structure (22 sections, 18 equations, 4 figures, 3 tables, 1 algorithm)

This paper contains 22 sections, 18 equations, 4 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Graph Contrastive Learning for Recommendation
  4. Alignment and Uniformity
  5. Methodology
  6. Graph Collaborative Filtering Backbone
  7. Prototypical Contrastive Learning
  8. Alignment and Uniformity Objectives
  9. Overall Loss
  10. Experiment
  11. Experiments Settings
  12. Datasets
  13. Baselines
  14. Evaluation Metrics
  15. Hyper-parameter
  16. ...and 7 more sections

Figures (4)

  • Figure 1: Illustration of sampling bias issue. The poor contrastive pair, which contains random negative samples with similar semantics to the original sample, would affect the performance of the recommendation system.
  • Figure 2: The overall framework of the proposed ProtoAU.
  • Figure 3: Visualizing study for the embeddings in recommendation systems on datasets Yelp2018.
  • Figure 4: Sensitivity analysis of the number of prototypes $K$ and the weights of alignment uniformity objectives $\lambda_2$, $\lambda_3$ on MovieLens-1M dataset with Recall10 results presentation.