Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

How Does Message Passing Improve Collaborative Filtering?

Mingxuan Ju, William Shiao, Zhichun Guo, Yanfang Ye, Yozen Liu, Neil Shah, Tong Zhao

TL;DR

Test-time Aggregation for CF is presented, namely TAG-CF, a test-time augmentation framework that only conducts message passing once at inference time that effectively utilizes graph knowledge while circumventing most of notorious computational overheads of message passing.

Abstract

Collaborative filtering (CF) has exhibited prominent results for recommender systems and been broadly utilized for real-world applications. A branch of research enhances CF methods by message passing used in graph neural networks, due to its strong capabilities of extracting knowledge from graph-structured data, like user-item bipartite graphs that naturally exist in CF. They assume that message passing helps CF methods in a manner akin to its benefits for graph-based learning tasks in general. However, even though message passing empirically improves CF, whether or not this assumption is correct still needs verification. To address this gap, we formally investigate why message passing helps CF from multiple perspectives and show that many assumptions made by previous works are not entirely accurate. With our curated ablation studies and theoretical analyses, we discover that (1) message passing improves the CF performance primarily by additional representations passed from neighbors during the forward pass instead of additional gradient updates to neighbor representations during the model back-propagation and (ii) message passing usually helps low-degree nodes more than high-degree nodes. Utilizing these novel findings, we present Test-time Aggregation for CF, namely TAG-CF, a test-time augmentation framework that only conducts message passing once at inference time. The key novelty of TAG-CF is that it effectively utilizes graph knowledge while circumventing most of notorious computational overheads of message passing. Besides, TAG-CF is extremely versatile can be used as a plug-and-play module to enhance representations trained by different CF supervision signals. Evaluated on six datasets, TAG-CF consistently improves the recommendation performance of CF methods without graph by up to 39.2% on cold users and 31.7% on all users, with little to no extra computational overheads.

How Does Message Passing Improve Collaborative Filtering?

TL;DR

Test-time Aggregation for CF is presented, namely TAG-CF, a test-time augmentation framework that only conducts message passing once at inference time that effectively utilizes graph knowledge while circumventing most of notorious computational overheads of message passing.

Abstract

Collaborative filtering (CF) has exhibited prominent results for recommender systems and been broadly utilized for real-world applications. A branch of research enhances CF methods by message passing used in graph neural networks, due to its strong capabilities of extracting knowledge from graph-structured data, like user-item bipartite graphs that naturally exist in CF. They assume that message passing helps CF methods in a manner akin to its benefits for graph-based learning tasks in general. However, even though message passing empirically improves CF, whether or not this assumption is correct still needs verification. To address this gap, we formally investigate why message passing helps CF from multiple perspectives and show that many assumptions made by previous works are not entirely accurate. With our curated ablation studies and theoretical analyses, we discover that (1) message passing improves the CF performance primarily by additional representations passed from neighbors during the forward pass instead of additional gradient updates to neighbor representations during the model back-propagation and (ii) message passing usually helps low-degree nodes more than high-degree nodes. Utilizing these novel findings, we present Test-time Aggregation for CF, namely TAG-CF, a test-time augmentation framework that only conducts message passing once at inference time. The key novelty of TAG-CF is that it effectively utilizes graph knowledge while circumventing most of notorious computational overheads of message passing. Besides, TAG-CF is extremely versatile can be used as a plug-and-play module to enhance representations trained by different CF supervision signals. Evaluated on six datasets, TAG-CF consistently improves the recommendation performance of CF methods without graph by up to 39.2% on cold users and 31.7% on all users, with little to no extra computational overheads.
Paper Structure (24 sections, 2 theorems, 13 equations, 4 figures, 9 tables)

This paper contains 24 sections, 2 theorems, 13 equations, 4 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Preliminaries and Related Work
  3. How Does Message Passing Improve Collaborative Filtering?
  4. Neighbor Information vs. Accompanying Gradients from Message Passing
  5. Message Passing in CF Helps Low-degree Users More Compared with High-degrees
  6. Test-time Aggregation for Collaborative Filtering
  7. Experiments
  8. Experimental Settings
  9. Performance Improvement to Matrix Factorization Methods
  10. Performance Comparison Among Graph-based Methods
  11. Effectiveness for Different Training Signals
  12. Performance w.r.t. User Degree
  13. Conclusion
  14. Limitation and Broader Impact
  15. Acknowledgments
  16. ...and 9 more sections

Key Result

Theorem 1

Assuming that $||\mathbf{u}_i||^2 = ||\mathbf{i}_j||^2 =1$ for any $u_i \in \mathcal{U}$ and $I_j \in \mathcal{I}$, objectives of BPR and DirectAU are strictly upper-bounded by the objective of message passing (i.e., $\mathcal{L}_{\text{BPR}} \leq \sum_{(i,j)\in \mathcal{D}} ||\mathbf{u}_i - \mathbf

Figures (4)

  • Figure 1: Performances of LightGCN and Matrix Factorization w.r.t. the user degree across datasets. The performance improvement brought by message passing decreases as the user degree goes up.
  • Figure 2: The performance and efficiency improvement of TAG-CF$^+$ w.r.t. different cutoffs. TAG-CF$^+$ further improves TAG-CF with less computational overheads. 100% is the original performance/efficiency of vanilla TAG-CF.
  • Figure 3: Improvement of TAG-CF$^+$ w.r.t. different cutoffs. Yellow dashed lines indicate TAG-CF, and black circles refer to the optimal degree cutoff that TAG-CF$^+$ selects.
  • Figure 4: The sensitivity of TAG-CF to $m$ and $n$ in \ref{['eq:tagcf']}. Numbers reported in these plots are performance improvement (%) brought by TAG-CF to MF trained by DirectAU wang2022towards on Recall@20.

Theorems & Definitions (4)

  • Theorem 1
  • Theorem 2
  • proof
  • proof