Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Off-Policy Evaluation for Recommendations with Missing-Not-At-Random Rewards

Tatsuki Takahashi, Chihiro Maru, Hiroko Shoji

TL;DR

The paper tackles offline/off-policy evaluation for recommender systems under dual biases: display position (missing rewards) and logging-policy bias, particularly when rewards are MNAR. It analyzes how standard OPE estimators can be biased in MNAR settings and introduces a novel MIPS-based estimator with reward-observation IPS (ROIPS) that leverages two propensity scores to achieve unbiased evaluation under a no-direct-effect assumption. The authors provide theoretical insights on MNAR-induced bias and unbiasedness of the ROIPS-enhanced estimator, plus synthetic experiments showing substantial MSE reductions when using ROIPS-based approaches, especially with heuristic ROIPS. This work offers a practical offline evaluation method that robustly handles missing-not-at-random rewards in biased logging data, enabling more reliable policy selection for recommender systems.

Abstract

Unbiased recommender learning (URL) and off-policy evaluation/learning (OPE/L) techniques are effective in addressing the data bias caused by display position and logging policies, thereby consistently improving the performance of recommendations. However, when both bias exits in the logged data, these estimators may suffer from significant bias. In this study, we first analyze the position bias of the OPE estimator when rewards are missing not at random. To mitigate both biases, we propose a novel estimator that leverages two probabilities of logging policies and reward observations as propensity scores. Our experiments demonstrate that the proposed estimator achieves superior performance compared to other estimators, even as the levels of bias in reward observations increases.

Off-Policy Evaluation for Recommendations with Missing-Not-At-Random Rewards

TL;DR

The paper tackles offline/off-policy evaluation for recommender systems under dual biases: display position (missing rewards) and logging-policy bias, particularly when rewards are MNAR. It analyzes how standard OPE estimators can be biased in MNAR settings and introduces a novel MIPS-based estimator with reward-observation IPS (ROIPS) that leverages two propensity scores to achieve unbiased evaluation under a no-direct-effect assumption. The authors provide theoretical insights on MNAR-induced bias and unbiasedness of the ROIPS-enhanced estimator, plus synthetic experiments showing substantial MSE reductions when using ROIPS-based approaches, especially with heuristic ROIPS. This work offers a practical offline evaluation method that robustly handles missing-not-at-random rewards in biased logging data, enabling more reliable policy selection for recommender systems.

Abstract

Unbiased recommender learning (URL) and off-policy evaluation/learning (OPE/L) techniques are effective in addressing the data bias caused by display position and logging policies, thereby consistently improving the performance of recommendations. However, when both bias exits in the logged data, these estimators may suffer from significant bias. In this study, we first analyze the position bias of the OPE estimator when rewards are missing not at random. To mitigate both biases, we propose a novel estimator that leverages two probabilities of logging policies and reward observations as propensity scores. Our experiments demonstrate that the proposed estimator achieves superior performance compared to other estimators, even as the levels of bias in reward observations increases.

Paper Structure

This paper contains 12 sections, 12 equations, 2 figures.

Table of Contents

  1. INTRODUCTION
  2. BACKGROUND AND RELATED WORK
  3. Unbiased recommender learning
  4. Off-policy evaluation/learning
  5. OUR PROPOSAL
  6. Bias of OPE estimator when rewards are MNAR
  7. MIPS with reward observation IPS
  8. SYNTHETIC EXPERIMENTS
  9. Setup
  10. Baselines
  11. Results
  12. CONCLUSION

Figures (2)

  • Figure 1: Scope of counterfactual modeling in our study
  • Figure 2: MSE, squared bias, and variance with varying levels of bias in reward observations. Note that, as shown in Eq.(4), the MSE (left) can be decomposed into the squared bias term (middle) and the variance term of the estimator (right).

Theorems & Definitions (2)

  • proof
  • proof