Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Epinet for Content Cold Start

Hong Jun Jeon, Songbin Liu, Yuantong Li, Jie Lyu, Hunter Song, Ji Liu, Peng Wu, Zheqing Zhu

TL;DR

This work tackles the exploration-exploitation challenge in content recommendation under content cold-start by employing Epinet-based approximate Thompson sampling to model epistemic uncertainty in neural models. The authors integrate an Epinet atop a base predictor to produce posterior samples efficiently, enabling online deployment in Facebook Reels during the retrieval stage. They demonstrate a 17% uplift in impressions and improvements in like rates and video completion, particularly for content with few prior impressions, indicating stronger exploration of new content. The study highlights practical benefits for scalable uncertainty quantification in large-scale online systems and suggests avenues for extending these ideas to ranking and reinforcement learning settings.

Abstract

The exploding popularity of online content and its user base poses an evermore challenging matching problem for modern recommendation systems. Unlike other frontiers of machine learning such as natural language, recommendation systems are responsible for collecting their own data. Simply exploiting current knowledge can lead to pernicious feedback loops but naive exploration can detract from user experience and lead to reduced engagement. This exploration-exploitation trade-off is exemplified in the classic multi-armed bandit problem for which algorithms such as upper confidence bounds (UCB) and Thompson sampling (TS) demonstrate effective performance. However, there have been many challenges to scaling these approaches to settings which do not exhibit a conjugate prior structure. Recent scalable approaches to uncertainty quantification via epinets have enabled efficient approximations of Thompson sampling even when the learning model is a complex neural network. In this paper, we demonstrate the first application of epinets to an online recommendation system. Our experiments demonstrate improvements in both user traffic and engagement efficiency on the Facebook Reels online video platform.

Epinet for Content Cold Start

TL;DR

This work tackles the exploration-exploitation challenge in content recommendation under content cold-start by employing Epinet-based approximate Thompson sampling to model epistemic uncertainty in neural models. The authors integrate an Epinet atop a base predictor to produce posterior samples efficiently, enabling online deployment in Facebook Reels during the retrieval stage. They demonstrate a 17% uplift in impressions and improvements in like rates and video completion, particularly for content with few prior impressions, indicating stronger exploration of new content. The study highlights practical benefits for scalable uncertainty quantification in large-scale online systems and suggests avenues for extending these ideas to ranking and reinforcement learning settings.

Abstract

The exploding popularity of online content and its user base poses an evermore challenging matching problem for modern recommendation systems. Unlike other frontiers of machine learning such as natural language, recommendation systems are responsible for collecting their own data. Simply exploiting current knowledge can lead to pernicious feedback loops but naive exploration can detract from user experience and lead to reduced engagement. This exploration-exploitation trade-off is exemplified in the classic multi-armed bandit problem for which algorithms such as upper confidence bounds (UCB) and Thompson sampling (TS) demonstrate effective performance. However, there have been many challenges to scaling these approaches to settings which do not exhibit a conjugate prior structure. Recent scalable approaches to uncertainty quantification via epinets have enabled efficient approximations of Thompson sampling even when the learning model is a complex neural network. In this paper, we demonstrate the first application of epinets to an online recommendation system. Our experiments demonstrate improvements in both user traffic and engagement efficiency on the Facebook Reels online video platform.

Paper Structure

This paper contains 16 sections, 11 equations, 6 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Problem Formulation
  4. Exploration for Content Cold Start
  5. Thompson Sampling
  6. Approximate Thompson Sampling with Epinet
  7. Epistemic Neural Networks
  8. Epinet
  9. Prior Networks
  10. Model Training
  11. Epinet Thompson Sampling Algorithm
  12. Recommendation Pipeline
  13. Experiments
  14. Implementation Details
  15. Experimental Results
  16. ...and 1 more sections

Figures (6)

  • Figure 1: The above diagram depicts the training setup of our recommendation system. Note that the epinet is part of the overarch component.
  • Figure 2: The above diagram depicts the content cold start funnel. Each source filters a large batch of video content and passes the results to a blender before it is ranked. Videos which achieve sufficient rank are presented to users and monitored for engagement. Our algorithm impacts the proposals of one of the sources which feed into the ranking algorithm.
  • Figure 3: We depict percentage change in like per impression between our method and the control. We group videos by their impression counts. We notice the most significant benefits in the videos with lower impression count which is promising for content cold start.
  • Figure 4: We depict percentage change in video view completion per impression between our method and the control. We group videos by their impression count. We notice improved video view completion across all impression counts.
  • Figure 5: We depict percentage change in watch score per impression between our method and the control. We group videos by their impression count.
  • ...and 1 more figures

Theorems & Definitions (3)

  • Example 1
  • Example 2
  • Example 3