Multiple Randomization Designs: Estimation and Inference with Interference

TL;DR

This paper develops a formal finite-sample theory for Simple Multiple Randomization Designs (SMRDs) in marketplaces with interference. It defines a two-population, buyer-seller framework and a local-interference assumption, derives unbiased estimators and exact variances for estimands that capture direct effects and spillovers, and proves a finite-population central limit theorem to enable design-based inference. It introduces SMRD as a tractable variant of MRDs, demonstrates how to detect spillovers and estimate average causal effects, and provides variance estimators and CLTs that generalize single-population results to a two-population setting. Simulations illustrate the practical gains of SMRDs in accounting for spillovers and improving inference, with extensions pointing to richer designs and more flexible interference structures for future work.

Abstract

Completely randomized experiments, originally developed by Fisher and Neyman in the 1930s, are still widely used in practice, even in online experimentation. However, such designs are of limited value for answering standard questions in marketplaces, where multiple populations of agents interact strategically, leading to complex patterns of spillover effects. In this paper, we derive the finite-sample properties of tractable estimators for "Simple Multiple Randomization Designs" (SMRDs), a new class of experimental designs which account for complex spillover effects in randomized experiments. Our derivations are obtained under a natural and general form of cross-unit interference, which we call "local interference". We discuss the estimation of main effects, direct effects, and spillovers, and present associated central limit theorems.

Figures (14)

• Figure 1: Distribution of $\widehat{\overline{\overline{Y}}}_{{\textcolor{red}{\rm cc}}}$ (left) and of the variance estimator $\widehat{\Sigma}_{{\textcolor{red}{\rm cc}}}$ (right). Black lines correspond to the population quantities ${\overline{\overline{y}}}_{{\textcolor{red}{\rm cc}}}$, $\mathop{\mathrm{\mathrm{Var}}}\nolimits\left(\widehat{\overline{\overline{Y}}}_{{\textcolor{red}{\rm cc}}}\right)$.
• Figure 2: Distribution of the estimator for the spillover effect $\hat{\tau}_{\rm{spill}}^B$ (left) and corresponding variance estimator $\widehat{\mathop{\mathrm{\mathrm{Var}}}\nolimits}^{\rm{hi}}(\hat{\tau}_{\rm{spill}}^B)$ (right). Black lines correspond to the population quantities.
• Figure 3: Left: distribution of the Studentized statistic $\widehat{\tau}_{\rm spill}^{\rm B}/\{\widehat{\mathop{\mathrm{\mathrm{Var}}}\nolimits}^{\rm hi}(\widehat{\tau}_{\rm spill}^{\rm B})\}^{1/2}$ and resulting conservative two-sided tests of the null hypothesis of no effect $H_0=\{\tau_{\rm spill}^{\rm B} = 0\}$ (statistics to the right of the black line reject $H_0$). Right: QQ plot comparing the Studentized statistic to a Gaussian law with the same mean and variance.
• Figure 4: Distribution of the standard difference-in-means estimator $\hat{\tau}_{\rm SRD}$ across 10,000 single randomized experiments (yellow), compared to the distribution of $\hat{\tau}_{\rm ATE}$ in as many SDRDs (red). The estimators are produced by re-drawing different randomization designs for the same underlying finite population, with potential outcomes given by \ref{['example:strategic-local-interference']}.
• Figure 5: Distribution of $\widehat{\overline{\overline{Y}}}_{{\textcolor{red}{\rm cc}}}$ (left) and of the variance estimator $\widehat{\Sigma}_{{\textcolor{red}{\rm cc}}}$ (right). Black lines are plotted in correspondence of the population quantities ${\overline{\overline{y}}}_{{\textcolor{red}{\rm cc}}}$, $\mathop{\mathrm{\mathrm{Var}}}\nolimits\left(\widehat{\overline{\overline{Y}}}_{{\textcolor{red}{\rm cc}}}\right)$.

Theorems & Definitions (105)

• Example 2.5
• Definition 3.1: Multiple Randomization Designs
• Definition 3.2: Homogeneous and Inhomogeneous Experiences
• Remark 3.3
• Definition 3.4: Simple Multiple Randomization Designs
• Lemma 3.5
• Lemma 4.1
• Theorem 4.2
• Theorem 4.3
• Theorem 4.4