Adaptive Sampling Policies Imply Biased Beliefs: A Generalization of the Hot Stove Effect
Jerker Denrell
TL;DR
This paper generalizes the hot stove effect to settings where negative beliefs reduce sampling rather than causing complete avoidance, showing that adaptive sampling yields biased final beliefs for a broad class of learning rules. It presents a two-period illustration and proves that if total sample size in period two is an increasing function of the first-period average, the final average is negatively biased, with analogous results for unimodal symmetric payoffs. The analysis extends to alternative learning models and demonstrates that even Bayesian updating can produce skewed belief distributions under adaptive sampling, though the average belief remains unbiased. The work highlights adaptive sampling as a parsimonious explanation for observed belief biases and discusses implications for learning, decision-making, and information processing in real-world contexts.
Abstract
The Hot Stove Effect is a negativity bias resulting from the adaptive character of learning. The mechanism is that learning algorithms that pursue alternatives with positive estimated values, but avoid alternatives with negative estimated values, will correct errors of overestimation but fail to correct errors of underestimation. Here, we generalize the theory behind the Hot Stove Effect to settings in which negative estimates do not necessarily lead to avoidance but to a smaller sample size (i.e., a learner selects fewer of alternative B if B is believed to be inferior but does not entirely avoid B). We formally demonstrate that the negativity bias remains in this set-up. We also show there is a negativity bias for Bayesian learners in the sense that most such learners underestimate the expected value of an alternative.