Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Wasserstein Dependent Graph Attention Network for Collaborative Filtering with Uncertainty

Haoxuan Li, Yuanxin Ouyang, Zhuang Liu, Wenge Rong, Zhang Xiong

TL;DR

This work addresses uncertainty in collaborative filtering by representing users and items as Gaussian embeddings and measuring distributional similarity with the 2-Wasserstein distance $W_2$. It introduces W-GAT, a graph attention framework that propagates Gaussian means and variances on a user-item graph and uses Wasserstein-based attention for interactions, paired with a Lipschitz-constrained Wasserstein-dependent mutual information term, with the objective $L = L_{BPR} + \omega L_{WPC} + \lambda (||\mu||^2 + ||\Sigma||^2)$. The approach yields improved recommendations on three real datasets, with notable gains on Digital_Music and robust performance on Movielens-1M and MIND, and provides insights into modeling uncertainty via variances corresponding to user interests and item categories. The contributions include analyzing KL-based MI limitations, proposing Wasserstein-dependent MI with Lipschitz constraints, and performing extensive ablations to demonstrate the necessity of L_WPC and the A^2 variance propagation for effective uncertainty modeling in graph-based CF.

Abstract

Collaborative filtering (CF) is an essential technique in recommender systems that provides personalized recommendations by only leveraging user-item interactions. However, most CF methods represent users and items as fixed points in the latent space, lacking the ability to capture uncertainty. While probabilistic embedding is proposed to intergrate uncertainty, they suffer from several limitations when introduced to graph-based recommender systems. Graph convolutional network framework would confuse the semantic of uncertainty in the nodes, and similarity measured by Kullback-Leibler (KL) divergence suffers from degradation problem and demands an exponential number of samples. To address these challenges, we propose a novel approach, called the Wasserstein dependent Graph Attention network (W-GAT), for collaborative filtering with uncertainty. We utilize graph attention network and Wasserstein distance to learn Gaussian embedding for each user and item. Additionally, our method incorporates Wasserstein-dependent mutual information further to increase the similarity between positive pairs. Experimental results on three benchmark datasets show the superiority of W-GAT compared to several representative baselines. Extensive experimental analysis validates the effectiveness of W-GAT in capturing uncertainty by modeling the range of user preferences and categories associated with items.

Wasserstein Dependent Graph Attention Network for Collaborative Filtering with Uncertainty

TL;DR

This work addresses uncertainty in collaborative filtering by representing users and items as Gaussian embeddings and measuring distributional similarity with the 2-Wasserstein distance . It introduces W-GAT, a graph attention framework that propagates Gaussian means and variances on a user-item graph and uses Wasserstein-based attention for interactions, paired with a Lipschitz-constrained Wasserstein-dependent mutual information term, with the objective . The approach yields improved recommendations on three real datasets, with notable gains on Digital_Music and robust performance on Movielens-1M and MIND, and provides insights into modeling uncertainty via variances corresponding to user interests and item categories. The contributions include analyzing KL-based MI limitations, proposing Wasserstein-dependent MI with Lipschitz constraints, and performing extensive ablations to demonstrate the necessity of L_WPC and the A^2 variance propagation for effective uncertainty modeling in graph-based CF.

Abstract

Collaborative filtering (CF) is an essential technique in recommender systems that provides personalized recommendations by only leveraging user-item interactions. However, most CF methods represent users and items as fixed points in the latent space, lacking the ability to capture uncertainty. While probabilistic embedding is proposed to intergrate uncertainty, they suffer from several limitations when introduced to graph-based recommender systems. Graph convolutional network framework would confuse the semantic of uncertainty in the nodes, and similarity measured by Kullback-Leibler (KL) divergence suffers from degradation problem and demands an exponential number of samples. To address these challenges, we propose a novel approach, called the Wasserstein dependent Graph Attention network (W-GAT), for collaborative filtering with uncertainty. We utilize graph attention network and Wasserstein distance to learn Gaussian embedding for each user and item. Additionally, our method incorporates Wasserstein-dependent mutual information further to increase the similarity between positive pairs. Experimental results on three benchmark datasets show the superiority of W-GAT compared to several representative baselines. Extensive experimental analysis validates the effectiveness of W-GAT in capturing uncertainty by modeling the range of user preferences and categories associated with items.
Paper Structure (28 sections, 1 theorem, 16 equations, 7 figures, 6 tables, 1 algorithm)

This paper contains 28 sections, 1 theorem, 16 equations, 7 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Preliminaries
  5. Gaussian Embeddings on Graph
  6. Wasserstein dependent GAT
  7. Wasserstein Dependent Mutual Information
  8. Limitations of Mutual Information by KL divergence
  9. Wasserstein dependent Mutual Information with Lipschitz Constraint
  10. Differences
  11. Training Details
  12. Complexity Analysis of W-GAT
  13. Experiments
  14. Experimental Settings
  15. Datasets
  16. ...and 13 more sections

Key Result

Theorem 1

Let p(x) and q(x) be two distributions and R and S be two sets of n samples from p(x) and q(x), respectively. Let $\delta$ be a confident parameter, and let B(R,S,$\delta$) be a real-valued function of the two samples S and R and the confidence parameter $\delta$. We have that, if with probability a then with probability at least $1-4\delta$, we have

Figures (7)

  • Figure 1: Differences between point embedding methods and Gaussian embedding methods.
  • Figure 2: The framework of W-GAT. We deploy W-GAT layers which propagate the means and variances of representations on the user-item bipartite graph. Then we use the in-batch sample strategy to select positive and negative pairs. Finally, we obtain the two loss functions based on the sampled user-item pairs.
  • Figure 3: The visualization of Gaussian embeddings of three different movies from M1M
  • Figure 4: Visualization of mean embedding learned by LightGCN and GAT on the M1M dataset.
  • Figure 5: The values of loss function with two different measurement under different batch sizes.
  • ...and 2 more figures

Theorems & Definitions (1)

  • Theorem 1