Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unifying Graph Convolution and Contrastive Learning in Collaborative Filtering

Yihong Wu, Le Zhang, Fengran Mo, Tianyu Zhu, Weizhi Ma, Jian-Yun Nie

TL;DR

It is shown that the graph convolutional layers often used in graph-based models are not essential for high-order connectivity modeling and might contribute to the risk of oversmoothing.

Abstract

Graph-based models and contrastive learning have emerged as prominent methods in Collaborative Filtering (CF). While many existing models in CF incorporate these methods in their design, there seems to be a limited depth of analysis regarding the foundational principles behind them. This paper bridges graph convolution, a pivotal element of graph-based models, with contrastive learning through a theoretical framework. By examining the learning dynamics and equilibrium of the contrastive loss, we offer a fresh lens to understand contrastive learning via graph theory, emphasizing its capability to capture high-order connectivity. Building on this analysis, we further show that the graph convolutional layers often used in graph-based models are not essential for high-order connectivity modeling and might contribute to the risk of oversmoothing. Stemming from our findings, we introduce Simple Contrastive Collaborative Filtering (SCCF), a simple and effective algorithm based on a naive embedding model and a modified contrastive loss. The efficacy of the algorithm is demonstrated through extensive experiments across four public datasets. The experiment code is available at \url{https://github.com/wu1hong/SCCF}. \end{abstract}

Unifying Graph Convolution and Contrastive Learning in Collaborative Filtering

TL;DR

It is shown that the graph convolutional layers often used in graph-based models are not essential for high-order connectivity modeling and might contribute to the risk of oversmoothing.

Abstract

Graph-based models and contrastive learning have emerged as prominent methods in Collaborative Filtering (CF). While many existing models in CF incorporate these methods in their design, there seems to be a limited depth of analysis regarding the foundational principles behind them. This paper bridges graph convolution, a pivotal element of graph-based models, with contrastive learning through a theoretical framework. By examining the learning dynamics and equilibrium of the contrastive loss, we offer a fresh lens to understand contrastive learning via graph theory, emphasizing its capability to capture high-order connectivity. Building on this analysis, we further show that the graph convolutional layers often used in graph-based models are not essential for high-order connectivity modeling and might contribute to the risk of oversmoothing. Stemming from our findings, we introduce Simple Contrastive Collaborative Filtering (SCCF), a simple and effective algorithm based on a naive embedding model and a modified contrastive loss. The efficacy of the algorithm is demonstrated through extensive experiments across four public datasets. The experiment code is available at \url{https://github.com/wu1hong/SCCF}. \end{abstract}
Paper Structure (35 sections, 4 theorems, 26 equations, 2 figures, 8 tables)

This paper contains 35 sections, 4 theorems, 26 equations, 2 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Task Definition
  4. Notation
  5. Graph Convolution
  6. Unifying Graph Convolution and Contrastive Learning
  7. Definition of Contrastive Loss
  8. Learning Dynamics of Contrastive Loss
  9. Equivalence of Contrastive Loss and Graph Convolution
  10. Equilibrium of Contrastive Learning
  11. Discussion
  12. High-Order Connectivity Modeling
  13. High-Order Connectivity from Contrastive Loss
  14. On The Necessity of Graph Convolutional layers
  15. A Brief Review of LightGCN
  16. ...and 20 more sections

Key Result

proposition 1

Given a graph and its corresponding Laplacian matrix $\mathbf{L}$, eigenvalues of $\mathbf{L}$ such that $\lambda_1 \leq \lambda_2 \leq \dots \leq \lambda_n$, $0< \gamma < 1/\lambda_n$, graph filter $\mathbf{I} - \gamma \mathbf{L}$ is a low-pass filter and graph filter $\mathbf{I}+\gamma\mathbf{L}$

Figures (2)

  • Figure 1: Performance with different temperatures on Amazon-Beauty and Yelp2018 dataset.
  • Figure 2: Embedding size vs. Recall@20 (left) and NDCG@20 (right) on the Gowalla dataset.

Theorems & Definitions (6)

  • definition 1: Graph Filter
  • definition 2: Graph Convolution
  • proposition 1
  • proposition 2
  • theorem 1
  • theorem 2