Correcting for Position Bias in Learning to Rank: A Control Function Approach

TL;DR

This work tackles position bias in implicit-feedback learning-to-rank by introducing a two-stage control-function framework (CFC) that leverages exogenous variation from the historical ranking to debias clicks without estimating propensities or altering the ranker's loss. Stage 1 residualizes the prior ranking to obtain ranking residuals, while Stage 2 uses these residuals (and feature interactions) as control terms in the click model, enabling bias-corrected predictions that can plug into any LTR algorithm. The authors also propose a debiased validation procedure for hyperparameter tuning when unbiased relevance labels are unavailable, and demonstrate through extensive experiments on three benchmarks that CFC outperforms state-of-the-art baselines under varying bias severities and data sparsity. The approach is flexible, robust, and extends to other biases, offering a practical, general solution for debiasing in real-world LTR systems.

Abstract

Implicit feedback data, such as user clicks, is commonly used in learning-to-rank (LTR) systems because it is easy to collect and it often reflects user preferences. However, this data is prone to various biases, and training an LTR algorithm directly on biased data can result in suboptimal ranking performance. One of the most prominent and well-studied biases in implicit feedback data is position bias, which occurs because users are more likely to interact with higher-ranked items regardless of their true relevance. In this paper, we propose a novel control function-based method that accounts for position bias in a two-stage process. The first stage uses exogenous variation from the residuals of the ranking process to correct for position bias in the second stage click equation. Unlike previous position bias correction methods, our method does not require knowledge of the click or propensity model and allows for nonlinearity in the underlying ranking model. Moreover, our method is general and allows for debiasing any state-of-the-art ranking algorithm by plugging it into the second stage. We also introduce a new technique to debias validation clicks for hyperparameter tuning to select the optimal model in the absence of unbiased validation data. Experimental results show that our method outperforms state-of-the-art approaches in correcting for position bias.

Figures (3)

• Figure 1: The performance of different methods with varying levels of position bias severity.
• Figure 2: The performance of different methods with varying the number of passes for clicks generation.
• Figure 3: The performance of different methods up to $\bm{10 \ (p)}$ positions with varying percentages of noisy clicks.