A prediction interval for the population-wise error rate
Remi Luschei, Werner Brannath
TL;DR
This work tackles controlling the population-wise error rate ($PWER$) across overlapping populations when strata prevalences are unknown. It derives an asymptotic prediction interval for the true $PWER$ using the delta method on the multinomial prevalence estimator, with the interval width determined by a variance term involving the gradient $\nabla_{\boldsymbol{\pi}} \text{PWER}$ and the covariance of $\hat{\boldsymbol{\pi}}$. The paper also introduces minimal-prevalence adjustments, compares bootstrap and $t$-approximation methods for the unknown-distribution case, and validates the approach with simulations and a real data example, showing generally accurate coverage and informative interval lengths. Practically, these prediction intervals provide a prognosis of $PWER$ control for future patients in umbrella or multi-population trials, and guidance on when to prefer bootstrap-based critical value estimation. The work thus offers a rigorous, data-informed tool for planning and interpreting multi-population clinical trials with overlapping populations.
Abstract
We construct an asymptotic prediction interval for the population-wise error rate (PWER), which is a multiple type I error criterion for clinical trials with overlapping patient populations. The PWER is the probability that a randomly selected patient will receive an ineffective treatment. It must usually be estimated due to unknown population strata sizes, such that only an estimate can be controlled at the given significance level. We apply the delta method to find a prediction interval for the resulting true PWER, we demonstrate by simulations that the interval has the required coverage probability, and illustrate the approach with real data examples.