Differences-in-Neighbors for Network Interference in Experiments

TL;DR

The paper tackles network interference in experiments by proposing Differences-in-Neighbors (DN), an estimator that reduces bias from interference while keeping variance tractable, especially when used with clustering. DN achieves second-order bias in the interference strength and variance that grows polynomially with network degree, offering a superior bias-variance tradeoff compared to naive DM and HT estimators. The authors develop a cluster-aware DN variant, prove theoretical guarantees including bias and variance bounds via Taylor expansions and smoothness assumptions, and demonstrate practical gains on small-world and real-world networks as well as a ridesharing simulator. The work bridges design and estimation choices, enabling smaller clusters and more accurate ATE estimation under complex interference patterns. Overall, DN extends the toolkit for scalable, robust causal inference in interconnected systems with practical implications for large online platforms.

Abstract

Experiments in online platforms frequently suffer from network interference, in which a treatment applied to a given unit affects outcomes for other units connected via the platform. This SUTVA violation biases naive approaches to experiment design and estimation. A common solution is to reduce interference by clustering connected units, and randomizing treatments at the cluster level, typically followed by estimation using one of two extremes: either a simple difference-in-means (DM) estimator, which ignores remaining interference; or an unbiased Horvitz-Thompson (HT) estimator, which eliminates interference at great cost in variance. Even combined with clustered designs, this presents a limited set of achievable bias variance tradeoffs. We propose a new estimator, dubbed Differences-in-Neighbors (DN), designed explicitly to mitigate network interference. Compared to DM estimators, DN achieves bias second order in the magnitude of the interference effect, while its variance is exponentially smaller than that of HT estimators. When combined with clustered designs, DN offers improved bias-variance tradeoffs not achievable by existing approaches. Empirical evaluations on a large-scale social network and a city-level ride-sharing simulator demonstrate the superior performance of DN in experiments at practical scale.

Figures (7)

• Figure 1: (Left) Temporal interference, where interference is captured through a Markov chain, is often considered a distinct class of problems from (Right) network interference, where interference is captured by interactions between nodes. On view of our work is a generalization of the Difference-in-Q estimators fariasMarkovianInterferenceExperiments2022fariasCorrectingInterferenceExperiments2023, which were proposed to address temporal interference in Markov chains, to network interference settings. This is achieved by observing that the Q-value is essentially the sum of (out-connected) neighbors in a general graph. This generalization enables a unifying view of temporal and network interference, allowing techniques developed for one domain to potentially benefit the other. The connection is formalized in \ref{['sec:outcome-model']}.
• Figure 2: Left: An example of a small-world network with parameters $N=20$, $d=4$, and $q=0.2$. Right: RMSE of different estimators while varying degree $d$. The graph has $N = 10000$ with rewiring probability $q = 0.05$. Each point in the plot represents the RMSE corresponding to the optimal cluster size that minimizes the RMSE for each estimator.
• Figure 3: Relative error with 95% confidence interval for small world random graph. We take the degree of the starting ring graph $d=20$, and set the rewiring probability $q= 0.1$. Again clusters $1$ to $8$ are produced with CPM clustering with resolution $0.005, 0.01, 0.05, 0.1,0.3, 0.5, 0.7, 0.9$. The rough size of the clusters ranges from $<10^2$ for resolutions $<0.1$, to around $10^2$ going up to $\sim600$ for resolutions $<0.7$, $10^3$ for resolution $0.7,0.9$, and Node size for $1.0$ and above.
• Figure 4: Relative bias with 95% confidence interval for Erdos-Renyi random graph. We take $p_{deg} = 20/N$, meaning the expected degree is $20$. Clusters $1$ to $9$ are produced with CPM clustering with resolution $0.005 0.01, 0.05, 0.1,0.3, 0.5, 0.7, 0.9, 1.0$.
• Figure 5: Real twitter network graph with synthetic outcome function. Clusters 1 through 5 correspond to resolution $0.0001$, $0.001$, $0.01$, $0.1$, $0.5$. Left: Relative error across $100$ trials. Right: RMSE and clustering information. DN's performance dominates DM regardless of the clustering scheme.

Theorems & Definitions (10)

• Theorem 1
• Corollary 1
• proof
• Theorem 2
• proof
• Theorem 3
• Theorem 4
• proof
• Lemma 1
• proof