The Surprising Benefits of Base Rate Neglect in Robust Aggregation

TL;DR

This work examines how base rate neglect (BRN) by experts affects the robustness of forecast aggregation with two binary-state experts. By introducing a BRN model with a prior-consideration parameter $\lambda$, the authors extend the robust regret framework to BRN settings and reveal a surprising V-shaped dependence of regret on $\lambda$, including cases where intermediate BRN yields lower regret than full Bayesian updating. They propose a closed-form family of $\hat{\lambda}$-BRN balancing aggregators that perform well across unknown $\lambda$ and demonstrate near-optimal performance (approx. $0.013$ regret) in simulations. An extensive online study with tens of thousands of information-structure instances confirms substantial BRN heterogeneity in humans and shows that BRN-aware aggregators can outperform non-BRN methods on BRN-skewed data, though simple averaging remains strong overall. The findings highlight that accounting for BRN can improve robustness in forecast aggregation and offer practical aggregators for decision-makers lacking full knowledge of signal structures.

Abstract

Robust aggregation integrates predictions from multiple experts without knowledge of the experts' information structures. Prior work assumes experts are Bayesian, providing predictions as perfect posteriors based on their signals. However, real-world experts often deviate systematically from Bayesian reasoning. Our work considers experts who tend to ignore the base rate. We find that a certain degree of base rate neglect helps with robust forecast aggregation. Specifically, we consider a forecast aggregation problem with two experts who each predict a binary world state after observing private signals. Unlike previous work, we model experts exhibiting base rate neglect, where they incorporate the base rate information to degree $λ\in[0,1]$, with $λ=0$ indicating complete ignorance and $λ=1$ perfect Bayesian updating. To evaluate aggregators' performance, we adopt Arieli et al. (2018)'s worst-case regret model, which measures the maximum regret across the set of considered information structures compared to an omniscient benchmark. Our results reveal the surprising V-shape of regret as a function of $λ$. That is, predictions with an intermediate incorporating degree of base rate $λ<1$ can counter-intuitively lead to lower regret than perfect Bayesian posteriors with $λ=1$. We additionally propose a new aggregator with low regret robust to unknown $λ$. Finally, we conduct an empirical study to test the base rate neglect model and evaluate the performance of various aggregators.

Figures (5)

• Figure 1: Our aggregator vs. Existing aggregators
• Figure 2: the Regret of $\hat{\lambda}$-base rate balancing aggregators $f^\lambda_{ap}$
• Figure 3: Proportions of Different Types of Response
• Figure 4: The Distribution of Estimated $\lambda_i$
• Figure 5: Aggregators' Performance in Subsample

Theorems & Definitions (12)

• Claim 1
• proof
• Claim 2: Alternative Formula for the Relative Loss arieli2018robust
• Theorem 1: Regret Curves Are Single-troughed
• proof : Proof of Theorem \ref{['theorem: V-shaped upper bound']}
• Lemma 1: Reduction of Regret Computation
• Claim 3
• Claim 4
• Lemma 2: Relative Loss Is Single-troughed
• Lemma 3: Supremum of Single-troughed Functions Is Single-troughed