Empirical Power Analysis of a Statistical Test to Quantify Gerrymandering

TL;DR

This paper empirically evaluates the power of the ε-outlier gerrymandering test (CFP/CFMP framework) on biased North Carolina district maps reconstructed from the 2012 and 2016 elections. It generates biased ensembles via hill climbing and short bursts on MCMC districting chains and analyzes how the test’s power varies with chain length, effect size, election year, party, and, crucially, the choice of partisan metric ω. The main finding is that power is largely determined by the metric used to bias the ensemble, with maximal power reachable at relatively small numbers of steps, and that Type I error remains controlled under typical settings; some metrics (e.g., efficiency gap) show different behavior under the null. The work provides replicable pipelines and open-source tools to assess and compare gerrymandering tests, contributing to the reliability of statistical evidence used in litigation and policy discussions in redistricting.

Abstract

Gerrymandering is a pervasive problem within the US political system. In the past decade, methods based on Markov Chain Monte Carlo (MCMC) sampling and statistical outlier tests have been proposed to quantify gerrymandering and were used as evidence in several high-profile legal cases. We perform an empirical power analysis of one such hypothesis test from Chikina et al (2020). We generate a family of biased North Carolina congressional district maps using the 2012 and 2016 presidential elections and assess under which conditions the outlier test fails to flag them at the specified Type I error level. The power of the outlier test is found to be relatively stable across political parties, election years, lengths of the MCMC chain and effect sizes. The main effect on the power of the test is shown to be the choice of the bias metric. This is the first work that computationally verifies the power of statistical tests used in gerrymandering cases.

Figures (12)

• Figure 2: Here, the Recombination algorithm is used to generate a new districting plan starting from the 2020 enacted plan in North Carolina.
• Figure 3: Examples of maps from biased chains. Biasing was done to favor the Republican party according to the mean median gap using 2016 Presidential election data. Coloring is arbitrary.
• Figure 4: Values of efficiency gaps in ensembles biased for Republicans and Democratics using hill climbing and short bursts respectively after 50000 steps for the 2016 Presidential Election in North Carolina.
• Figure 5: Histograms of mean median gap values from biased NC maps. Biased chains are run to maximize mean median gap with hill-climbing (left, $N=50000$) and short burst (right, $N=10000, k=5$) using 2016 Presidential election data. Chains start at the map in Fig. \ref{['fig:original_map']}, whose mean median gap value is marked in solid vertical line. Blue is distribution of maps biased for Democrats, and red is for Republicans. Gray is for neutral Recom chain of length 50000 and are the same in both subplots.
• Figure 6: Scatter plots indicate nonlinear relationships between bias metrics. Histograms of mean median gap values from biased NC maps. Blue points are maps biased for Democrats, and red is for Republicans. Gray is for neutral Recom chain. In each subplot, horizontal axis is the metric the chain was biased towards, and vertical axis is the metric we are measuring. When the metric used to bias the chains and the metric measured from the resulting chains are the same, we plot the histograms as in Fig. \ref{['fig:biased_mean_median']}. Biased chains were generated via short burst (Alg. \ref{['alg:shortburst']}) for $N=10000, k=5$ using 2016 presidential election data. For readability, every 50th map in the chain was sampled for the scatter plots.

Theorems & Definitions (3)

• theorem 1
• Definition 1.1.1: ($\epsilon,\alpha$)-outlier chikina2020separating
• theorem 2