Understanding and Counteracting Feature-Level Bias in Click-Through Rate Prediction

TL;DR

This work identifies feature-level bias in CTR prediction and pinpoints the linear feature projection component as the primary driver. It conducts a top-down analysis showing that imbalanced positive sample ratios across item groups bias linear weights, which in turn bias predictions and recommendations. To counteract this with minimal disruption, the authors propose two post-training strategies: linear weight reduction, and linear weight reconstruction that leverages limited unbiased data, achieving improved item-side fairness and maintained or enhanced accuracy across three datasets. The findings offer a practical, low-overhead path to fairer CTR systems and illuminate how training data imbalances propagate through linear model components.

Abstract

Common click-through rate (CTR) prediction recommender models tend to exhibit feature-level bias, which leads to unfair recommendations among item groups and inaccurate recommendations for users. While existing methods address this issue by adjusting the learning of CTR models, such as through additional optimization objectives, they fail to consider how the bias is caused within these models. To address this research gap, our study performs a top-down analysis on representative CTR models. Through blocking different components of a trained CTR model one by one, we identify the key contribution of the linear component to feature-level bias. We conduct a theoretical analysis of the learning process for the weights in the linear component, revealing how group-wise properties of training data influence them. Our experimental and statistical analyses demonstrate a strong correlation between imbalanced positive sample ratios across item groups and feature-level bias. Based on this understanding, we propose a minimally invasive yet effective strategy to counteract feature-level bias in CTR models by removing the biased linear weights from trained models. Additionally, we present a linear weight adjusting strategy that requires fewer random exposure records than relevant debiasing methods. The superiority of our proposed strategies are validated through extensive experiments on three real-world datasets.

Figures (7)

• Figure 1: A toy example to illustrate feature-level bias. (a) Ideal and Actual denote the expected number of recommendation by ground truth and by CTR model, respectively. (b) Item-side unfairness, group A and group B are over-recommended and under-recommended, respectively. (c) Hurting user experiences, the model recommends 60% A and 40% B to a user (ID=23), whereas his true preference consists of 40% A and 60% B.
• Figure 2: An example to illustrate the calculation of EHR. Circle, diamond and triangle denote three different item groups. GT denotes the ground-truth of user interactions. Rec denotes the ranked user list given by the CTR model.
• Figure 3: (a) Illustration of item-side unfairness w.r.t. $P(j,K=5)$. (b) An example of the gap between History and True Preferences and the amplification effect of the CTR model. History denotes the ratio of positive samples in a group to the total positive samples collected from previous recommendations. CTR Model denotes the ratio of predicted positive samples in a group to the total positive samples collected from previous recommendations. True Preferences denotes the ratio of positive samples in a group to the total positive samples collected from the random exposure.
• Figure 4: Relations between linear weights and three statistics. Each point denotes an item feature group in ML-1M and the dashed lines are obtained by linear regression.
• Figure 5: Illustration of the generation path of feature-level bias in a NFM model. In (a), each item group's positive sample ratios are sorted in ascending order. In (b)(c), we calculate the Pearson pearson correlation coefficients (denoted as $PS$) between the value of interested ($y$-axis) and the positive sample ratio to verify their linear relationship. In (c), "pos" and "neg" denote the average prediction scores computed over all positive or negative testing samples, respectively. In (d), $SP$ and p-value are results of Spearman test spearman between each item group's $EHR$ in Equation \ref{['eq:ratio']} and positive sample ratio, showing the monotonic relationship between them.