Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

(Debiased) Contrastive Learning Loss for Recommendation (Technical Report)

Ruoming Jin, Dong Li

TL;DR

A systemic examination of the recommendation losses, including listwise (softmax), pairwise (BPR), and pointwise (mean-squared error, MSE, and Cosine Contrastive Loss, CCL) losses through the lens of contrastive learning shows the effectiveness of the debiased losses and newly introduced mutual-information losses outperform the existing (biased) ones.

Abstract

In this paper, we perform a systemic examination of the recommendation losses, including listwise (softmax), pairwise(BPR), and pointwise (mean-squared error, MSE, and Cosine Contrastive Loss, CCL) losses through the lens of contrastive learning. We introduce and study both debiased InfoNCE and mutual information neural estimator (MINE), for the first time, under the recommendation setting. We also relate and differentiate these two losses with the BPR loss through the lower bound analysis. Furthermore, we present the debiased pointwise loss (for both MSE and CCL) and theoretically certify both iALS and EASE, two of the most popular linear models, are inherently debiased. The empirical experimental results demonstrate the effectiveness of the debiased losses and newly introduced mutual-information losses outperform the existing (biased) ones.

(Debiased) Contrastive Learning Loss for Recommendation (Technical Report)

TL;DR

A systemic examination of the recommendation losses, including listwise (softmax), pairwise (BPR), and pointwise (mean-squared error, MSE, and Cosine Contrastive Loss, CCL) losses through the lens of contrastive learning shows the effectiveness of the debiased losses and newly introduced mutual-information losses outperform the existing (biased) ones.

Abstract

In this paper, we perform a systemic examination of the recommendation losses, including listwise (softmax), pairwise(BPR), and pointwise (mean-squared error, MSE, and Cosine Contrastive Loss, CCL) losses through the lens of contrastive learning. We introduce and study both debiased InfoNCE and mutual information neural estimator (MINE), for the first time, under the recommendation setting. We also relate and differentiate these two losses with the BPR loss through the lower bound analysis. Furthermore, we present the debiased pointwise loss (for both MSE and CCL) and theoretically certify both iALS and EASE, two of the most popular linear models, are inherently debiased. The empirical experimental results demonstrate the effectiveness of the debiased losses and newly introduced mutual-information losses outperform the existing (biased) ones.
Paper Structure (31 sections, 2 theorems, 43 equations, 3 figures, 5 tables)

This paper contains 31 sections, 2 theorems, 43 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Contrastive Recommendation Loss
  3. Basic (Sampled) Softmax and InfoNCE loss
  4. Leveraging Debiased Contrastive InfoNCE
  5. Connection with MINE and BPR
  6. Debiased Pointwise Loss
  7. Debiased MSE and CCL
  8. iALS, EASE and their Debiasness
  9. Experiment
  10. Experimental Setup
  11. Datasets
  12. Evaluation Metrics
  13. Models
  14. Loss functions
  15. Reproducibility
  16. ...and 16 more sections

Key Result

Theorem 1

For any debiased iALS loss $\mathcal{L}^{Debiased}_{iALS}$ with parameters $\alpha_0$ and $\lambda$ with constant $c_u$ for all users, there are original iALS loss with parameters $\alpha_0^\prime$ and $\lambda^\prime$, which have the same closed form solutions (up to a constant factor) for fixing i

Figures (3)

  • Figure 1: Effect of number of positive and negative samples on $Gowalla$
  • Figure 2: Effect of negative weight on $Gowalla$
  • Figure 3: Effect of temperature on $Gowalla$

Theorems & Definitions (4)

  • Definition 1
  • Definition 2
  • Theorem 1
  • Theorem 2