Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Ranking Across Different Content Types: The Robust Beauty of Multinomial Blending

Jan Malte Lichtenberg, Giuseppe Di Benedetto, Matteo Ruffini

TL;DR

A simple method for cross-content-type ranking, called multinomial blending (MB), is explored, which can be used in conjunction with most existing LTR algorithms and compared to existing baselines not only in terms of ranking quality but also from other industry-relevant perspectives.

Abstract

An increasing number of media streaming services have expanded their offerings to include entities of multiple content types. For instance, audio streaming services that started by offering music only, now also offer podcasts, merchandise items, and videos. Ranking items across different content types into a single slate poses a significant challenge for traditional learning-to-rank (LTR) algorithms due to differing user engagement patterns for different content types. We explore a simple method for cross-content-type ranking, called multinomial blending (MB), which can be used in conjunction with most existing LTR algorithms. We compare MB to existing baselines not only in terms of ranking quality but also from other industry-relevant perspectives such as interpretability, ease-of-use, and stability in dynamic environments with changing user behavior and ranking model retraining. Finally, we report the results of an A/B test from an Amazon Music ranking use-case.

Ranking Across Different Content Types: The Robust Beauty of Multinomial Blending

TL;DR

A simple method for cross-content-type ranking, called multinomial blending (MB), is explored, which can be used in conjunction with most existing LTR algorithms and compared to existing baselines not only in terms of ranking quality but also from other industry-relevant perspectives.

Abstract

An increasing number of media streaming services have expanded their offerings to include entities of multiple content types. For instance, audio streaming services that started by offering music only, now also offer podcasts, merchandise items, and videos. Ranking items across different content types into a single slate poses a significant challenge for traditional learning-to-rank (LTR) algorithms due to differing user engagement patterns for different content types. We explore a simple method for cross-content-type ranking, called multinomial blending (MB), which can be used in conjunction with most existing LTR algorithms. We compare MB to existing baselines not only in terms of ranking quality but also from other industry-relevant perspectives such as interpretability, ease-of-use, and stability in dynamic environments with changing user behavior and ranking model retraining. Finally, we report the results of an A/B test from an Amazon Music ranking use-case.
Paper Structure (9 sections, 2 figures, 1 table)

This paper contains 9 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. The Cross-Content-Type (CCT) ranking problem
  3. Multinomial Blending (MB)
  4. Experiments
  5. Discussion and Conclusion
  6. Appendix
  7. MMR selection for ranking diversification.
  8. MB with "at least $p$%" exposure guarantees in the $C=2$ case.
  9. MB propensity estimation

Figures (2)

  • Figure 1: Multinomial blending (MB) for $C=3$ content types (red, blue, gray) and a ranking size of $k=4$. At each step, the blender samples a content type according to sampling probabilities $\bm{p} = [p_1, p_2, p_3]$ and then selects the highest-scoring remaining candidate. In this example, the candidate pool for the red content type is empty after the third iteration (sub-figure c), leading to re-normalization of the sampling probabilities for remaining content types.
  • Figure 2: MMR selection for cross-content-type ranking and $C=3$ content types (red, blue, gray). After each action selection, the scores of the remaining candidate items are penalized by their content-type similarity to items that have already been ranked.