Sensitivity Analysis for Binary Outcome Misclassification in Randomization Tests via Integer Programming
Siyu Heng, Pamela A. Shaw
TL;DR
This work tackles how binary outcome misclassification can bias randomization tests in finite populations. It introduces warning accuracy (WA), a model-free threshold that quantifies how much misclassification could overturn a causal conclusion, and shows how to compute WA via adaptive integer programming that respects the randomization design. The authors develop two reformulations—P1 for Type I designs with within-strata symmetry and P2 for Type II designs with between-strata symmetry—to erase symmetry and enable scalable computation under Fisher's sharp null and Neyman's weak null. They apply the framework to the Prostate Cancer Prevention Trial (PCPT), deriving WA values and sensitivity weights that illuminate which misclassification types most threaten the conclusions, and they provide an open-source R package RIOM for practical use.
Abstract
Conducting a randomization test is a common method for testing causal null hypotheses in randomized experiments. The popularity of randomization tests is largely because their statistical validity only depends on the randomization design, and no distributional or modeling assumption on the outcome variable is needed. However, randomization tests may still suffer from other sources of bias, among which outcome misclassification is a significant one. We propose a model-free and finite-population sensitivity analysis approach for binary outcome misclassification in randomization tests. A central quantity in our framework is ``warning accuracy," defined as the threshold such that a randomization test result based on the measured outcomes may differ from that based on the true outcomes if the outcome measurement accuracy did not surpass that threshold. We show how learning the warning accuracy and related concepts can amplify analyses of randomization tests subject to outcome misclassification without adding additional assumptions. We show that the warning accuracy can be computed efficiently for large data sets by adaptively reformulating a large-scale integer program with respect to the randomization design. We apply the proposed approach to the Prostate Cancer Prevention Trial (PCPT). We also developed an open-source R package for implementation of our approach.