Predicting Juror Predisposition Using Machine Learning: A Comparative Study of Human and Algorithmic Jury Selection

TL;DR

This study benchmarks juror prediction by directly comparing professional jury consultants to supervised ML models (Random Forest and KNN) using identical pre-trial questionnaire data. In a controlled mock-trial design with 410 participants (273 train, 137 test), ML models significantly outperform consultant majority votes, achieving accuracies of $0.818$ (RF) and $0.796$ (KNN) versus $0.693$ for humans, with robust bootstrap and McNemar tests confirming significance. Attitudinal signals predominantly drive predictions, while demographic features contribute little and including race/gender does not materially alter results, highlighting a distributed, interpretable attitudinal basis for prediction. The authors frame ML as a transparent, auditable decision-support benchmark that can improve accountability in voir dire without replacing human judgment, and they emphasize the need for fairness auditing and legal scrutiny before any real-world deployment.

Abstract

Prior studies on the effectiveness of professional jury consultants in predicting juror proclivities have yielded mixed results, and few have rigorously evaluated consultant performance against chance under controlled conditions. This study addresses that gap by empirically assessing whether jury consultants can reliably predict juror predispositions beyond chance levels and whether supervised machine-learning (ML) models can outperform consultant predictions. Using data from N mock jurors who completed pre-trial attitudinal questionnaires and rendered verdicts in a standardized wrongful-termination case, we compared predictions made by professional jury consultants with those generated by Random Forest (RF) and k-Nearest Neighbors (KNN) classifiers. Model and consultant predictions were evaluated on a held-out test set using paired statistical tests and nonparametric bootstrap procedures. We find that supervised ML models significantly outperform professional jury consultants under identical informational constraints, while offering greater transparency, replicability, and auditability. These results provide an empirical benchmark for evaluating human judgment in jury selection and inform ongoing debates about the role of data-driven decision support in legal contexts. To support reproducibility and auditability, all code and data will be made publicly available upon publication.

Figures (8)

• Figure 1: Comparison of predictive accuracy between professional jury consultants (majority vote) and machine-learning models. Error bars denote 95% paired bootstrap confidence intervals relative to the human baseline.
• Figure 2: Error overlap patterns across human consultants, Random Forest, and k-Nearest Neighbors models. Each bar represents the number of jurors corresponding to a specific error pattern, where 1 indicates an incorrect prediction and 0 indicates a correct prediction (ordered as Human, RF, KNN).
• Figure 3: Row-normalized confusion matrices comparing the human jury consultant majority vote, Random Forest, and k-Nearest Neighbors predictions on the held-out test set. Values indicate the proportion of jurors within each true class assigned to each predicted class, enabling direct comparison of error patterns across decision systems.
• Figure 4: Aggregate feature importance by feature type for the Random Forest model. Attitudinal variables account for the majority of predictive importance, while demographic attributes contribute substantially less.
• Figure 5: Top predictive features identified by the Random Forest model, ranked by Gini importance. The most influential predictors correspond to jurors’ attitudinal beliefs about workplace fairness, discrimination, and responsibility attribution, while demographic attributes exhibit comparatively low importance.