Trimming of extreme votes and favoritism: Evidence from the field

TL;DR

The paper investigates whether trimming extreme votes reduces bias in subjective expert evaluations by exploiting a real-world natural experiment in Olympic-level freeski halfpipe judging. Using 29,383 judge scores across periods before and after the introduction of a trimming rule (highest and lowest scores removed), and employing run fixed effects and judge-season controls, the authors identify a clear nationalistic bias prior to trimming that diminishes substantially after trimming. The analysis shows that top-score shares for compatriot judges were notably higher than the unbiased baseline before trimming but align more closely with unbiased levels after trimming, with fixed effects results indicating a large reduction in bias and often non-significant net bias post-trimming. The findings suggest that trimming not only mitigates ex post bias by discarding extreme votes but also ex ante influences evaluator behavior toward more sincere scoring, highlighting trimming as an effective and practical policy for improving panel evaluations in competitive contexts.

Abstract

Despite a large body of theoretical literature on voting mechanisms, there is no documented evidence from real-world panel evaluations about the effect of trimming the extreme votes on sincere voting. We provide the first such evidence by comparing subjective evaluations of experts from different countries in competitive settings with and without a trimming mechanism. In these evaluations, some of the evaluated subjects are experts' compatriots. Using data on 29,383 subjective evaluations, we find that experts assign significantly higher scores to their compatriots in panels without trimming. However, in panels with trimming, this favoritism is generally insignificant.

Figures (2)

• Figure 1: Example of competition results without and with trimming as presented in the official results documents
• Figure 2: Score level distribution for compatriot judges