Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unveiling Contrastive Learning's Capability of Neighborhood Aggregation for Collaborative Filtering

Yu Zhang, Yiwen Zhang, Yi Zhang, Lei Sang, Yun Yang

TL;DR

The Light Contrastive Collaborative Filtering (LightCCF) method is proposed, which introduces a novel neighborhood aggregation objective to bring users closer to all interacted items while pushing them away from other positive pairs, thus achieving high-quality neighborhood aggregation with very low time complexity.

Abstract

Personalized recommendation is widely used in the web applications, and graph contrastive learning (GCL) has gradually become a dominant approach in recommender systems, primarily due to its ability to extract self-supervised signals from raw interaction data, effectively alleviating the problem of data sparsity. A classic GCL-based method typically uses data augmentation during graph convolution to generates more contrastive views, and performs contrast on these new views to obtain rich self-supervised signals. Despite this paradigm is effective, the reasons behind the performance gains remain a mystery. In this paper, we first reveal via theoretical derivation that the gradient descent process of the CL objective is formally equivalent to graph convolution, which implies that CL objective inherently supports neighborhood aggregation on interaction graphs. We further substantiate this capability through experimental validation and identify common misconceptions in the selection of positive samples in previous methods, which limit the potential of CL objective. Based on this discovery, we propose the Light Contrastive Collaborative Filtering (LightCCF) method, which introduces a novel neighborhood aggregation objective to bring users closer to all interacted items while pushing them away from other positive pairs, thus achieving high-quality neighborhood aggregation with very low time complexity. On three highly sparse public datasets, the proposed method effectively aggregate neighborhood information while preventing graph over-smoothing, demonstrating significant improvements over existing GCL-based counterparts in both training efficiency and recommendation accuracy. Our implementations are publicly accessible.

Unveiling Contrastive Learning's Capability of Neighborhood Aggregation for Collaborative Filtering

TL;DR

The Light Contrastive Collaborative Filtering (LightCCF) method is proposed, which introduces a novel neighborhood aggregation objective to bring users closer to all interacted items while pushing them away from other positive pairs, thus achieving high-quality neighborhood aggregation with very low time complexity.

Abstract

Personalized recommendation is widely used in the web applications, and graph contrastive learning (GCL) has gradually become a dominant approach in recommender systems, primarily due to its ability to extract self-supervised signals from raw interaction data, effectively alleviating the problem of data sparsity. A classic GCL-based method typically uses data augmentation during graph convolution to generates more contrastive views, and performs contrast on these new views to obtain rich self-supervised signals. Despite this paradigm is effective, the reasons behind the performance gains remain a mystery. In this paper, we first reveal via theoretical derivation that the gradient descent process of the CL objective is formally equivalent to graph convolution, which implies that CL objective inherently supports neighborhood aggregation on interaction graphs. We further substantiate this capability through experimental validation and identify common misconceptions in the selection of positive samples in previous methods, which limit the potential of CL objective. Based on this discovery, we propose the Light Contrastive Collaborative Filtering (LightCCF) method, which introduces a novel neighborhood aggregation objective to bring users closer to all interacted items while pushing them away from other positive pairs, thus achieving high-quality neighborhood aggregation with very low time complexity. On three highly sparse public datasets, the proposed method effectively aggregate neighborhood information while preventing graph over-smoothing, demonstrating significant improvements over existing GCL-based counterparts in both training efficiency and recommendation accuracy. Our implementations are publicly accessible.

Paper Structure

This paper contains 32 sections, 19 equations, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Problem Definition
  4. Graph and Graph Convolution
  5. Contrastive Learning
  6. Investigation of Contrastive Learning in Recommendation
  7. Exploring InfoNCE Gradient Descent
  8. Effectiveness of InfoNCE and GCN Encoder
  9. Neighborhood Aggregation in InfoNCE
  10. Methodology
  11. LightCCF
  12. Multi-Task Joint Training
  13. Discussion
  14. Necessity of Graph Convolution
  15. Relation with SCCF
  16. ...and 17 more sections

Figures (4)

  • Figure 1: Sampling strategies for neighborhood aggregation (NA) loss. Left: positive pairs are user-item pairs, and negative pairs are the item set. Right: positive pairs are user-item pairs, and negative pairs are other user-item pairs.
  • Figure 2: Performance comparison w.r.t. NDCG@20 of LightCCF and other models for different user groups sparsity levels on Douban-book, Tmall, and Amazon-book Datasets.
  • Figure 3: Performance comparison w.r.t. noise ratio. The bar represents Recall, while the line represents the percentage of performance degradation across two Datasets.
  • Figure 4: Hyperparameter sensitivities to the temperature coefficient $\tau$ and the graph contrastive regularization weight $\alpha$w.r.t. Recall@20 across three datasets.