Minimizing Live Experiments in Recommender Systems: User Simulation to Evaluate Preference Elicitation Policies

TL;DR

The paper tackles onboarding in recommender systems by replacing or augmenting costly live experiments with a production-integrated simulator built on counterfactually robust user models. It develops two core components—static/latent user-state generation and in-session response modeling—trained on real onboarding data and coupled to production infrastructure to emulate end-to-end onboarding. Empirically, the approach reproduces key live metrics, supports off-policy evaluation for unseen policies, and enables offline optimization that aligns with post-launch outcomes, thereby reducing cycle time and user exposure to experimental policies. The work highlights practical benefits and challenges in broadening simulation-based evaluation across RS domains, including model robustness and reliability concerns.

Abstract

Evaluation of policies in recommender systems typically involves A/B testing using live experiments on real users to assess a new policy's impact on relevant metrics. This ``gold standard'' comes at a high cost, however, in terms of cycle time, user cost, and potential user retention. In developing policies for ``onboarding'' new users, these costs can be especially problematic, since on-boarding occurs only once. In this work, we describe a simulation methodology used to augment (and reduce) the use of live experiments. We illustrate its deployment for the evaluation of ``preference elicitation'' algorithms used to onboard new users of the YouTube Music platform. By developing counterfactually robust user behavior models, and a simulation service that couples such models with production infrastructure, we are able to test new algorithms in a way that reliably predicts their performance on key metrics when deployed live. We describe our domain, our simulation models and platform, results of experiments and deployment, and suggest future steps needed to further realistic simulation as a powerful complement to live experiments.

Figures (6)

• Figure 1: The artist selection interface. Users select artists they like, and skip those they don't. The scrollable interface allows for selection of as many artists as desired, with "Done" confirming the end of the onboarding session. The artists displayed as the user scrolls are selected dynamically given earlier selections and non-selections (or skips).
• Figure 2: Histograms of cosine similarity between inferred user interests and selected/unselected artists (left). Artist click-through rate with respect to cosine similarity (right).
• Figure 3: Convergence of training the user context generator: histogram of log-probabilities over number of training steps (left) and Wasserstein distance between the ground truth distribution of the number of artists in inferred user interests and the generated one (right).
• Figure 4: Precision (left) and recall (right) when comparing generated user interests with ground-truth user interests.
• Figure 5: Cumulative distribution functions (CDFs) for both session lengths (left) and number of selections per session (right) by interacting with the data-generating policy where x-axis represents the rank in a batch of 4,096 sessions. Generated CDFs are red and observed CDFs in the log are black.