Multiple conditional randomization tests for lagged and spillover treatment effects

Abstract

We consider the problem of constructing multiple independent conditional randomization tests using a single dataset. Because the tests are independent, the randomization p-values can be interpreted individually and combined using standard methods for multiple testing. We give a simple, sequential construction of such tests, and then discuss its application to three problems: Rosenbaum's evidence factors for observational studies, lagged treatment effect in stepped-wedge trials, and spillover effect in randomized trials with interference. We compare the proposed approach with some existing methods using simulated and real datasets. Finally, we establish a more general sufficient condition for independent conditional randomization tests.

Figures (7)

• Figure 1: Illustration of CRTs for lagged effect (lag $l=1$) in a stepped-wedge trial.
• Figure 2: Effect estimates from MCRTs+Z and mixed-effects models with and without time effect parameters: 90%-confidence intervals (CIs) of lagged effects on real data collected from four different stepped-wedge randomized trials.
• Figure 3: An illustration of the static unit selection procedure in \ref{['eq:Ik']}. Panel (a) shows the units' locations. In panel (b), two green focal units $j\in \mathcal{J}$ has some red neighbours in its $\epsilon^{(1)}$-ball, which will be included in $\mathcal{I}^{(1)}$. In panel (c), we fix the treatment assignment of $\mathcal{I}^{(1)}$ (i.e. exclude them from $\mathcal{I}^{(2)}$) when testing the spillover effect at a larger distance $\epsilon^{(2)}$; the green focal unit on the left has no red neighbour as the distance increases.
• Figure 4: The subsets of units $\mathcal{I}^{(k)}$ (red) and $\mathcal{J}^{(k)}$ (green) created by the baseline method (fixing the same subset of focal units in all tests) and our sequential covering procedure.
• Figure 5: Performance of MCRTs+F, MCRTs+Z and Bonferroni's method: type I error rates and powers in testing lagged effects at five different numbers of units, numbers of time steps, time lags and effect sizes. The results were averaged over 1000 independent runs.

Theorems & Definitions (24)

• Example 1: Evidence factors
• Example 2: Lagged treatment effect
• Example 3: Testing the range of spillover effect
• Definition 1
• Theorem 1
• Theorem 2
• proof
• Theorem 3
• Proposition 1
• Proposition 2