Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bias or Optimality? Disentangling Bayesian Inference and Learning Biases in Human Decision-Making

Prakhar Godara

TL;DR

This work finds that both confirmation bias and unbiased but decreasing learning rates yield the same behavioral signatures, and proposes experimental protocols to disentangle true cognitive biases from artifacts of decreasing learning rates.

Abstract

Recent studies claim that human behavior in a two-armed Bernoulli bandit (TABB) task is described by positivity and confirmation biases, implying that humans do not integrate new information objectively. However, we find that even if the agent updates its belief via objective Bayesian inference, fitting the standard Q-learning model with asymmetric learning rates still recovers both biases. Bayesian inference cast as an effective Q-learning algorithm has symmetric, though decreasing, learning rates. We explain this by analyzing the stochastic dynamics of these learning systems using master equations. We find that both confirmation bias and unbiased but decreasing learning rates yield the same behavioral signatures. Finally, we propose experimental protocols to disentangle true cognitive biases from artifacts of decreasing learning rates.

Bias or Optimality? Disentangling Bayesian Inference and Learning Biases in Human Decision-Making

TL;DR

This work finds that both confirmation bias and unbiased but decreasing learning rates yield the same behavioral signatures, and proposes experimental protocols to disentangle true cognitive biases from artifacts of decreasing learning rates.

Abstract

Recent studies claim that human behavior in a two-armed Bernoulli bandit (TABB) task is described by positivity and confirmation biases, implying that humans do not integrate new information objectively. However, we find that even if the agent updates its belief via objective Bayesian inference, fitting the standard Q-learning model with asymmetric learning rates still recovers both biases. Bayesian inference cast as an effective Q-learning algorithm has symmetric, though decreasing, learning rates. We explain this by analyzing the stochastic dynamics of these learning systems using master equations. We find that both confirmation bias and unbiased but decreasing learning rates yield the same behavioral signatures. Finally, we propose experimental protocols to disentangle true cognitive biases from artifacts of decreasing learning rates.
Paper Structure (8 sections, 37 equations, 4 figures, 1 table)

This paper contains 8 sections, 37 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Explaining the observation
  3. Action switching under constant learning rates
  4. Action switching under time-varying learning rates
  5. Consequences on human behavior
  6. Model comparison
  7. Potential empirical investigations
  8. Discussion

Figures (4)

  • Figure 1: (a) Ensemble averaged best fit learning rates when fit to human data palminteri2017 (blue) and Bayes-optimal agents (green). (b) Ensemble averaged action switching rates $\langle K \rangle_t$ as a function of time for $Q$-learning algorithms (black), humans (blue) and Bayes-optimal policy (green).
  • Figure 2: Steady state $\Delta^*$ as a function of confirmation bias $x$, for different values of (a) $\beta$ and (b) $p$ values. (c) shows the average action switching probability $\langle K \rangle_t$ as a function of $t$, for different values of confirmation bias.
  • Figure 3: (a) The minimum (across time $t$) value of the average action switching rate $\langle K\rangle_{\text{min}}$ as a function of $\tau_c$ for different values of $\alpha_1$. (b) $\alpha_t$ and the corresponding action switching rates $\langle K\rangle_t$ as a function for time for $\tau_c = 25$ and $\alpha_1,\alpha_2 = 0.1,0.01$.
  • Figure 4: (a) Ensemble average action switching rates for human data (blue), best fit Bayesian agents (green), best fit $Q$-learning agents (black) for both unbiased (dashed) and biased (dotted) learning rates. (b) Distribution of best describing models for all human subjects by BIC (solid) scores and NLL (striped) scores. (c) Best fit $Q$-value trajectories for an examplar human subject corresponding to Bayesian (green) and $Q$-learning model (black). (d) Probability of choosing the newly introduced arm as a function of it's reward rate, for the best fit Bayesian (green) and $Q$-learning model (black).