Clustered Switchback Designs for Experimentation Under Spatio-temporal Interference

TL;DR

The paper addresses causal experimentation under simultaneous spatio-temporal interference and non-stationarity by proposing a clustered switchback design and a radius-$r$ truncated Horvitz-Thompson estimator. It proves an $\tilde{O}(1/(NT))$ MSE bound (up to a graph-dependent factor) for graphs with good clustering, unifying results for pure switchback and A/B testing under interference. The framework uses a Markovian state evolution with rapid mixing and derives explicit bias and variance bounds via a dependence graph, with results specialized to sparse, $\kappa$-restricted growth, and spatial graphs. Simulations across single- and multi-unit settings validate the theoretical rates and demonstrate practical advantages of clustering both in space and time.

Abstract

We consider experimentation in the presence of non-stationarity, inter-unit (spatial) interference, and carry-over effects (temporal interference), where we wish to estimate the global average treatment effect (GATE), the difference between average outcomes having exposed all units at all times to treatment or to control. We suppose spatial interference is described by a graph, where a unit's outcome depends on its neighborhood's treatments, and that temporal interference is described by an MDP, where the transition kernel under either treatment (action) satisfies a rapid mixing condition. We propose a clustered switchback design, where units are grouped into clusters and time steps are grouped into blocks, and each whole cluster-block combination is assigned a single random treatment. Under this design, we show that for graphs that admit good clustering, a truncated Horvitz-Thompson estimator achieves a $\tilde O(1/NT)$ mean squared error (MSE), matching the lower bound up to logarithmic terms for sparse graphs. Our results simultaneously generalize the results from \citet{hu2022switchback,ugander2013graph} and \citet{leung2022rate}. Simulation studies validate the favorable performance of our approach.

Figures (12)

• Figure 1: Clustered Switchback Experiments. The image illustrates clustered switchback on DoorDash sneider19. The time and geographical locations are grouped into blocks. Each spatio-temporal cluster (i.e., product set) is independently assigned treatment/control. The goal is to estimate the difference in the average (counterfactual) "outcomes" (e.g., revenues) between the all-treatment and all-control policy..
• Figure 2: Positioning of this work. By and large, a model for experimentation involves a subset of four key features, as illustrated above. Prior work has addressed spatial and temporal interference separately, often incorporating heterogeneity across individuals as well. However, in real-world applications, all four features often arise simultaneously. This work aims to lay the theoretical foundation for cluster switchback experiments, which are increasingly used in practice to navigate such complex interference.
• Figure 3: Dependence Graph: The regions correspond to the clusters in a partition $\Pi$. Units $i,j$ intersect a common cluster $C$, so $(i,j)\in E_\Pi$ (or $i\not \perp\!\!\!\perp j$).
• Figure 4: (a) MSE, Stationary
• Figure 5: (b) MSE, Non-stationary

Theorems & Definitions (32)

• Definition 1: Global Average Treatment Effect
• Proposition 1: Initial State Doesn't Matter
• Definition 2: Clusters
• Definition 3: Clustered Switchback Design
• Definition 4: Radius-$r$ Truncated Horvitz-Thompson (HT) Estimator
• Remark 1
• Definition 5: Dependence Graph
• Lemma 1: Independence for Far-apart Individuals
• Proposition 2: Bias of the HT estimator
• Definition 6: Cluster Degree