Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Probabilistic Rank and Reward: A Scalable Model for Slate Recommendation

Imad Aouali, Achraf Ait Sidi Hammou, Otmane Sakhi, David Rohde, Flavian Vasile

TL;DR

Probabilistic Rank and Reward (PRR) introduces a scalable model for slate recommendations where a user can interact with at most one item from a slate of size $K$. It jointly models reward and rank via a categorical distribution with engagement features $\mathbf{y}$ and user-interest features $\mathbf{z}$, enabling fast decision making through maximum inner product search (MIPS). The decision rule reduces to a MIPS problem by removing nuisance factors, yielding end-to-end $\mathcal{O}(\log P)$ retrieval suitable for large catalogs and low-latency domains. Empirical results on synthetic and session-completion tasks show PRR outperforms off-policy reward baselines (IPS/DM) and variants, while offering superior computational efficiency and scalability. The work highlights a practical, model-based approach that bridges reward optimization and ranking in slate settings and points to extensions with priors and richer interaction modeling.

Abstract

We introduce Probabilistic Rank and Reward (PRR), a scalable probabilistic model for personalized slate recommendation. Our approach allows off-policy estimation of the reward in the scenario where the user interacts with at most one item from a slate of K items. We show that the probability of a slate being successful can be learned efficiently by combining the reward, whether the user successfully interacted with the slate, and the rank, the item that was selected within the slate. PRR outperforms existing off-policy reward optimizing methods and is far more scalable to large action spaces. Moreover, PRR allows fast delivery of recommendations powered by maximum inner product search (MIPS), making it suitable in low latency domains such as computational advertising.

Probabilistic Rank and Reward: A Scalable Model for Slate Recommendation

TL;DR

Probabilistic Rank and Reward (PRR) introduces a scalable model for slate recommendations where a user can interact with at most one item from a slate of size . It jointly models reward and rank via a categorical distribution with engagement features and user-interest features , enabling fast decision making through maximum inner product search (MIPS). The decision rule reduces to a MIPS problem by removing nuisance factors, yielding end-to-end retrieval suitable for large catalogs and low-latency domains. Empirical results on synthetic and session-completion tasks show PRR outperforms off-policy reward baselines (IPS/DM) and variants, while offering superior computational efficiency and scalability. The work highlights a practical, model-based approach that bridges reward optimization and ranking in slate settings and points to extensions with priors and richer interaction modeling.

Abstract

We introduce Probabilistic Rank and Reward (PRR), a scalable probabilistic model for personalized slate recommendation. Our approach allows off-policy estimation of the reward in the scenario where the user interacts with at most one item from a slate of K items. We show that the probability of a slate being successful can be learned efficiently by combining the reward, whether the user successfully interacted with the slate, and the rank, the item that was selected within the slate. PRR outperforms existing off-policy reward optimizing methods and is far more scalable to large action spaces. Moreover, PRR allows fast delivery of recommendations powered by maximum inner product search (MIPS), making it suitable in low latency domains such as computational advertising.
Paper Structure (23 sections, 14 equations, 4 figures, 2 tables, 4 algorithms)

This paper contains 23 sections, 14 equations, 4 figures, 2 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Inverse Propensity Scoring (IPS)
  4. Direct Methods (DM)
  5. Policy Learning
  6. Decision Making
  7. Summary of Limitations
  8. Algorithm
  9. Setting
  10. Modeling Rank and Reward
  11. Decision Making
  12. Experiments
  13. Goals
  14. Baselines
  15. Synthetic Problems
  16. ...and 8 more sections

Figures (4)

  • Figure 1: Example of 3 slates of size 2 on a technology website. From left to right are good, mixed and bad recommendations. $\bar{R}, r_1, r_2$ denote the probabilities of no-click, click on the $1$st and $2$nd item, respectively.
  • Figure 2: A diagram of the PRR model.
  • Figure 3: The reward (y-axis) of methods in synthetic problems with varying slate sizes (x-axis), number of items (columns) and logging policies (rows). The shaded areas represent uncertainty and they are small since we run long A/B tests with $n_{\rm test}=100{\rm k}$.
  • Figure 4: On the left-hand side, we report the reward (y-axis) of methods in session completion problems with varying slate sizes (x-axis). On the right-hand side, we report the training time (y-axis) of PRR and baselines in session completion problems with varying catalog sizes (x-axis).