A/B testing under Interference with Partial Network Information

TL;DR

This work tackles the challenge of estimating the Global Average Treatment Effect $\tau(\vec{1},\vec{0})$ in A/B tests when the exact interference graph is unknown but a superset of neighbors is available. It introduces UNITE, a principled estimation framework with a linear/additive interference model and extensions to non-linear motif-based interactions, alongside self-normalized and doubly robust variants for variance reduction. The paper proves unbiasedness under partial neighborhood containment, provides variance bounds that decay as $O(1/n)$, and demonstrates asymptotic normality to enable Wald-type confidence intervals. Empirical results on synthetic Erdos-Renyi graphs and an Airbnb-like case study show that UNITE achieves accurate, efficient GATE estimation without requiring exact network knowledge, highlighting its practical relevance for privacy-preserving analyses and epidemic-control interventions.

Abstract

A/B tests are often required to be conducted on subjects that might have social connections. For e.g., experiments on social media, or medical and social interventions to control the spread of an epidemic. In such settings, the SUTVA assumption for randomized-controlled trials is violated due to network interference, or spill-over effects, as treatments to group A can potentially also affect the control group B. When the underlying social network is known exactly, prior works have demonstrated how to conduct A/B tests adequately to estimate the global average treatment effect (GATE). However, in practice, it is often impossible to obtain knowledge about the exact underlying network. In this paper, we present UNITE: a novel estimator that relax this assumption and can identify GATE while only relying on knowledge of the superset of neighbors for any subject in the graph. Through theoretical analysis and extensive experiments, we show that the proposed approach performs better in comparison to standard estimators.

Figures (7)

• Figure 1: $Z=1$ denotes to units in the treatment group and $Z=0$ denotes units in the control group. (a) Standard A/B testing where there is no interaction between the treatment and the control units.(b) Network interference due to (unknown) interaction between the units. (c) We do not assume access to the exact underlying interaction graph. Instead, we consider a practically feasible assumption where only a superset of neighbors for any node is available.
• Figure 2: Examples of network motifs. Red square represent a node under consideration, and circles are its neighbours. the left column lists possible subgraphs of the neighbourhood of a node. The right hand side depicts different motifs corresponding to these subgraphs. The yellow node represents $Z=0$ while purple represents $Z=1$. A motif is activated when its nodes are assigned the corresponding treatment assignment
• Figure 3: Performance of GATE estimators under Bernoulli design on Erdos-Renyi networks. Y-axes represent the relative value of the absolute bias i.e. $\left| \frac{\mathbb{E}[\hat{\tau} - \tau(\vec{1},\vec{0})]}{\tau} \right|$, with the shaded width corresponding to the experimental deviation. The rows correspond to linear, quadratic and sigmoid model for the potential outcomes. Columns correspond to different parameters being varied: (a) Strength of interference, where the x-axis corresponds to the average ratio of indirect to direct effects $r = \frac{1}{n}\sum_{i,j}\frac{|c_{ij}|}{{c_{ii}}|}$. (b) Population size, where the x-axis corresponds to the number of nodes $n$ (c) Treatment budget, where the x-axis corresponds to the probability of treatment 1 ($p$).
• Figure 4: Visualization of relative bias and relative RMSE of different GATE estimators as the indirect treatment effect $\alpha$ increases.
• Figure 5: Visualization of the impact of neighbourhood sizes on GATE estimation on the AirBnb Study. Negative fraction of neighbours indicate the case when $\mathcal{M}(i) \subset \mathcal{N}_i$ i.e. we missed pertinent neighbours. The bias tends to be high when given small neighborhoods, as they miss pertinent edges. As the neighborhood sizes increase, the bias reduces, but the uncertainty widens.

Theorems & Definitions (35)

• Remark
• Theorem 1
• Remark 1
• Remark 2
• Theorem 2
• Theorem 3
• Theorem 4
• Theorem 5
• Theorem 6
• Theorem 7