Can Probabilistic Feedback Drive User Impacts in Online Platforms?

TL;DR

This work reveals that even perfectly aligned learning objectives cannot guarantee positive user welfare, because probabilistic feedback can steer a recommender system toward promoting content with certain properties. By modeling content selection as a multi-armed bandit with arm-specific feedback rates, the authors quantify engagement via APC and FOC and introduce the concepts of feedback monotonicity and balance. They propose three black-box transformations (BB_Divide, BB_Pull, BB_DA) to study how probabilistic feedback shapes arm engagement and provide regret guarantees, supplemented by refined analyses and an empirical study of correlations between feedback rates and downstream effects. The results highlight the need to evaluate algorithms beyond regret, considering how feedback dynamics influence content exposure and user experience, with implications for platform design and policy.

Abstract

A common explanation for negative user impacts of content recommender systems is misalignment between the platform's objective and user welfare. In this work, we show that misalignment in the platform's objective is not the only potential cause of unintended impacts on users: even when the platform's objective is fully aligned with user welfare, the platform's learning algorithm can induce negative downstream impacts on users. The source of these user impacts is that different pieces of content may generate observable user reactions (feedback information) at different rates; these feedback rates may correlate with content properties, such as controversiality or demographic similarity of the creator, that affect the user experience. Since differences in feedback rates can impact how often the learning algorithm engages with different content, the learning algorithm may inadvertently promote content with certain such properties. Using the multi-armed bandit framework with probabilistic feedback, we examine the relationship between feedback rates and a learning algorithm's engagement with individual arms for different no-regret algorithms. We prove that no-regret algorithms can exhibit a wide range of dependencies: if the feedback rate of an arm increases, some no-regret algorithms engage with the arm more, some no-regret algorithms engage with the arm less, and other no-regret algorithms engage with the arm approximately the same number of times. From a platform design perspective, our results highlight the importance of looking beyond regret when measuring an algorithm's performance, and assessing the nature of a learning algorithm's engagement with different types of content as well as their resulting downstream impacts.

Figures (4)

• Figure 1: Analysis of $\mathtt{APC}\xspace$ for a simplified version of 3-Phase EXP3 (Algorithm \ref{['algo:EXP3-3phase']}) in two instances where $K=2$ and $T=1000$. In Instance 1 (left), Arm 1 has constant loss $0.9$ and Arm 2 has constant loss $0.1$; In Instance 2 (right), Arm 1 has constant loss $0.1$ and Arm 2 has constant loss $0.9$. $\mathtt{APC}\xspace$ is strictly negative monotonic in Instance 1 and strictly positive in Instance 2. These differing directions of monotonicity suggest that Algorithm \ref{['algo:EXP3-3phase']} does not exhibit clean monotonicity guarantees.
• Figure 2: Correlations induced between $f_i$ and $\mathtt{APC}\xspace_i$ (top row) as well as $\mathtt{FOC}\xspace_i$ (bottom row) by $\text{\upshape BB}_{\text{\upshape Pull}}(\textsc{AAE})$ (left column), $\text{\upshape BB}_{\text{\upshape Pull}}(\textsc{UCB})$ (middle), and 3-Phase EXP3 (right). There are $K = 100$ arms and $T = 1000$ rounds. The darkness of a point indicates the corresponding arm's average utility; darker is higher.
• Figure 3: Timelines of $\text{\upshape BB}_{\text{\upshape Pull}}(\textsc{Alg})$ on instances $\cal I$ (top row) and $\widetilde{\mathcal{I}}$ (bottom row) are demonstrated. Each time step $t\in [T]$ maps to a block number in $\mathcal{I}$ that is no more than its block number in $\widetilde{\mathcal{I}}$. The total number of times $\textsc{Alg}$ is called in instance $\mathcal{I}$, $\Phi$, and the number of times it is called in $\widetilde{\mathcal{I}}$, $\widetilde{\Phi}$, satisfy $\Phi\leq \widetilde{\Phi}$.
• Figure 4: Timelines of $\text{\upshape BB}_{\text{\upshape DA}}(\textsc{Alg})$ on instances $\cal I$ (top row) and $\widetilde{\mathcal{I}}$ (bottom row) are demonstrated. Each time step $t\in [T]$ maps to a block number in $\mathcal{I}$ that is no less than its block number in $\widetilde{\mathcal{I}}$. The total number of times $\textsc{Alg}$ is called in instance $\mathcal{I}$, $\Phi$, and the number of times it is called in $\widetilde{\mathcal{I}}$, $\widetilde{\Phi}$, satisfy $\Phi\geq \widetilde{\Phi}$. Note that this is similar to \ref{['fig:bbpull-time']}, except that the direction of monotonicity has switched and that the size of $B_i$ and $\widetilde{B}_i$ is deterministic in each instance.

Theorems & Definitions (91)

• Example 1: Own-group content and $\mathtt{APC}\xspace$
• Example 2: Incendiary content and $\mathtt{FOC}\xspace$.
• Definition 1: Regret
• Definition 2: Arm Pull Count ($\mathtt{APC}\xspace$)
• Definition 3: Feedback Observation Count ($\mathtt{FOC}\xspace$)
• Lemma 2.0
• Definition 4: Feedback monotonicity.
• Definition 5: Balance
• Proposition 2.0
• Theorem 3.1: Regret $\BBDivide$