Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Revisiting Recommendation Loss Functions through Contrastive Learning (Technical Report)

Dong Li, Ruoming Jin, Bin Ren

TL;DR

InformationNCE+, an optimized generalization of InfoNCE with balance coefficients, is introduced, and its performance advantages are highlighted, particularly when aligned with the new decoupled contrastive loss, MINE+.

Abstract

Inspired by the success of contrastive learning, we systematically examine recommendation losses, including listwise (softmax), pairwise (BPR), and pointwise (MSE and CCL) losses. In this endeavor, we introduce InfoNCE+, an optimized generalization of InfoNCE with balance coefficients, and highlight its performance advantages, particularly when aligned with our new decoupled contrastive loss, MINE+. We also leverage debiased InfoNCE to debias pointwise recommendation loss (CCL) as Debiased CCL. Interestingly, our analysis reveals that linear models like iALS and EASE are inherently debiased. Empirical results demonstrates the effectiveness of MINE+ and Debiased-CCL.

Revisiting Recommendation Loss Functions through Contrastive Learning (Technical Report)

TL;DR

InformationNCE+, an optimized generalization of InfoNCE with balance coefficients, is introduced, and its performance advantages are highlighted, particularly when aligned with the new decoupled contrastive loss, MINE+.

Abstract

Inspired by the success of contrastive learning, we systematically examine recommendation losses, including listwise (softmax), pairwise (BPR), and pointwise (MSE and CCL) losses. In this endeavor, we introduce InfoNCE+, an optimized generalization of InfoNCE with balance coefficients, and highlight its performance advantages, particularly when aligned with our new decoupled contrastive loss, MINE+. We also leverage debiased InfoNCE to debias pointwise recommendation loss (CCL) as Debiased CCL. Interestingly, our analysis reveals that linear models like iALS and EASE are inherently debiased. Empirical results demonstrates the effectiveness of MINE+ and Debiased-CCL.
Paper Structure (23 sections, 2 theorems, 36 equations, 4 figures, 5 tables)

This paper contains 23 sections, 2 theorems, 36 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Decoupled Contrastive Recommendation Loss
  4. Generalized Contrastive Learning: InfoNCE+
  5. Simplified Contrastive Learning: MINE+
  6. Lower Bound Analysis
  7. Debiased Contrastive Recommendation Loss
  8. Debiased CCL
  9. iALS and EASE are debiased
  10. Experiment
  11. Experimental Setup
  12. Q1. Performance of MINE/MINE+
  13. Q2. (Biased) Loss vs Debiased Loss
  14. Q3. Hyperparameter analysis
  15. conclusion
  16. ...and 8 more sections

Key Result

Theorem 1

For any debiased iALS loss $\mathcal{L}^{Debiased}_{iALS}$ with parameters $\alpha_0$ and $\lambda$ with constant $c_u$ for all users, there are original iALS loss with parameters $\alpha_0^\prime$ and $\lambda^\prime$, which have the same closed form solutions (up to a constant factor) for fixing i

Figures (4)

  • Figure 1: Empirical results on $\epsilon$ tuning. ($\lambda$ is fixed according to performance, typically around 1.1.)
  • Figure 2: Effect of number of samples on $Gowalla$
  • Figure 3: Effect of negative weight $\lambda_0$ on debiased InfoNCE loss $Gowalla$
  • Figure 4: Effect of noise weight $\lambda$ in MINE+ loss on $Gowalla$

Theorems & Definitions (7)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Theorem 1
  • proof
  • Theorem 2