Improved Estimation of Ranks for Learning Item Recommenders with Negative Sampling

TL;DR

This work provides sampled batch version of the well-studied WARP and LambdaRank methods and demonstrates that WARP and LambdaRank can be learned efficiently with negative sampling and the proposed correction technique.

Abstract

In recommendation systems, there has been a growth in the number of recommendable items (# of movies, music, products). When the set of recommendable items is large, training and evaluation of item recommendation models becomes computationally expensive. To lower this cost, it has become common to sample negative items. However, the recommendation quality can suffer from biases introduced by traditional negative sampling mechanisms. In this work, we demonstrate the benefits from correcting the bias introduced by sampling of negatives. We first provide sampled batch version of the well-studied WARP and LambdaRank methods. Then, we present how these methods can benefit from improved ranking estimates. Finally, we evaluate the recommendation quality as a result of correcting rank estimates and demonstrate that WARP and LambdaRank can be learned efficiently with negative sampling and our proposed correction technique.

Figures (2)

• Figure 1: Evaluation of batched versions of WARP and LambdaRank with different negative sample sizes.
• Figure 2: Effect of correction at various thresholds k of $\mathrm{NDCG}@k$ for LambdaRank.