Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MBD: A Model-Based Debiasing Framework Across User, Content, and Model Dimensions

Yuantong Li, Lei Yuan, Zhihao Zheng, Weimiao Wu, Songbin Liu, Jeong Min Lee, Ali Selman Aydin, Shaofeng Deng, Junbo Chen, Xinyi Zhang, Hongjing Xia, Sam Fieldman, Matthew Kosko, Wei Fu, Du Zhang, Peiyu Yang, Albert Jin Chung, Xianlei Qiu, Miao Yu, Zhongwei Teng, Hao Chen, Sunny Baek, Hui Tang, Yang Lv, Renze Wang, Qifan Wang, Zhan Li, Tiantian Xu, Peng Wu, Ji Liu

Abstract

Modern recommendation systems rank candidates by aggregating multiple behavioral signals through a value model. However, many commonly used signals are inherently affected by heterogeneous biases. For example, watch time naturally favors long-form content, loop rate favors short - form content, and comment probability favors videos over images. Such biases introduce two critical issues: (1) value model scores may be systematically misaligned with users' relative preferences - for instance, a seemingly low absolute like probability may represent exceptionally strong interest for a user who rarely engages; and (2) changes in value modeling rules can trigger abrupt and undesirable ecosystem shifts. In this work, we ask a fundamental question: can biased behavioral signals be systematically transformed into unbiased signals, under a user - defined notion of ``unbiasedness'', that are both personalized and adaptive? We propose a general, model-based debiasing (MBD) framework that addresses this challenge by augmenting it with distributional modeling. By conditioning on a flexible subset of features (partial feature set), we explicitly estimate the contextual mean and variance of the engagement distribution for arbitrary cohorts (e.g., specific video lengths or user regions) directly alongside the main prediction. This integration allows the framework to convert biased raw signals into unbiased representations, enabling the construction of higher-level, calibrated signals (such as percentiles or z - scores) suitable for the value model. Importantly, the definition of unbiasedness is flexible and controllable, allowing the system to adapt to different personalization objectives and modeling preferences. Crucially, this is implemented as a lightweight, built-in branch of the existing MTML ranking model, requiring no separate serving infrastructure.

MBD: A Model-Based Debiasing Framework Across User, Content, and Model Dimensions

Abstract

Modern recommendation systems rank candidates by aggregating multiple behavioral signals through a value model. However, many commonly used signals are inherently affected by heterogeneous biases. For example, watch time naturally favors long-form content, loop rate favors short - form content, and comment probability favors videos over images. Such biases introduce two critical issues: (1) value model scores may be systematically misaligned with users' relative preferences - for instance, a seemingly low absolute like probability may represent exceptionally strong interest for a user who rarely engages; and (2) changes in value modeling rules can trigger abrupt and undesirable ecosystem shifts. In this work, we ask a fundamental question: can biased behavioral signals be systematically transformed into unbiased signals, under a user - defined notion of ``unbiasedness'', that are both personalized and adaptive? We propose a general, model-based debiasing (MBD) framework that addresses this challenge by augmenting it with distributional modeling. By conditioning on a flexible subset of features (partial feature set), we explicitly estimate the contextual mean and variance of the engagement distribution for arbitrary cohorts (e.g., specific video lengths or user regions) directly alongside the main prediction. This integration allows the framework to convert biased raw signals into unbiased representations, enabling the construction of higher-level, calibrated signals (such as percentiles or z - scores) suitable for the value model. Importantly, the definition of unbiasedness is flexible and controllable, allowing the system to adapt to different personalization objectives and modeling preferences. Crucially, this is implemented as a lightweight, built-in branch of the existing MTML ranking model, requiring no separate serving infrastructure.
Paper Structure (30 sections, 13 equations, 5 figures, 4 tables)

This paper contains 30 sections, 13 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Preliminary: Existing Systems
  3. Signal bias widely exist in typical recommendation systems
  4. Value Model (VM).
  5. Inherent Biases in Signal Aggregation.
  6. Naive Debiasing Approaches via Bucketized Counting
  7. Limitations.
  8. MBD: Model-Based Debiasing Framework
  9. Specifying Bias
  10. Contextual Mean and Variance Estimates on Bias Set via Supervised Learning
  11. Unbiased Signal Construction from Contextual Mean and Variance
  12. Experiments
  13. Offline Evaluation
  14. Contextual Distribution Estimation Quality
  15. Bias Mitigation
  16. ...and 15 more sections

Figures (5)

  • Figure 1: The MBD framework augments the MTML model with an additional branch that estimates contextual distribution statistics $(\mu, \sigma^2)$ through a partial feature set.
  • Figure 2: Transformed watch time ($P_{50}$) trend over one month bucketed by video length.
  • Figure 3: User-level transformed time spent mean and variance distribution stratified by stories length.
  • Figure 4: Analysis of MBD predictions across duration buckets: (a) The predicted mean $\mu_{\text{MBD}}$ closely tracks the empirical distribution $p(\mathbf{x})$; (b) The predicted variance $\sigma_{\text{MBD}}^2$ effectively captures the uncertainty within each bucket.
  • Figure 5: Analysis of MBD predictions for Likes in logit space: (a) The predicted mean $\mu_{\text{MBD}}$ accurately tracks the main model's logit prediction $p(\mathbf{x})$; (b) The predicted variance $\sigma_{\text{MBD}}^2$ quantifies the uncertainty of user preference intensity.

Theorems & Definitions (1)

  • Definition 3.1: Bias Feature Set