Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Estimating Total Effects in Bipartite Experiments with Spillovers and Partial Eligibility

Albert Tan, Mohsen Bayati, James Nordlund, Roman Istomin

TL;DR

This work addresses estimation of total effects in eligibility-constrained bipartite experiments where only a subset of treatment-side units can be randomized, but interference propagates across all units. It introduces Primary Total Treatment Effect (PTTE) and Secondary Total Treatment Effect (STTE) and develops ensemble estimators that fuse exposure mappings, generalized propensity scores, and machine learning, with a projection linking treatment- and outcome-level estimands under a Linear Additive Edges assumption. The projection enables efficient estimation at the treatment level with aggregation to outcomes, yielding accurate PTTE and STTE estimates in simulations and two real-field experiments, and highlights how ignoring spillovers can bias decision metrics. The framework provides practical guidance for deploying interference-aware evaluations in bipartite systems and points to directions for handling partially observed networks, dynamic settings, and non-additive outcomes, with significant implications for rollout planning in platform economies.

Abstract

We study randomized experiments in bipartite systems where only a subset of treatment-side units are eligible for assignment while all units continue to interact, generating interference. We formalize eligibility-constrained bipartite experiments and define estimands aligned with full deployment: the Primary Total Treatment Effect (PTTE) on eligible units and the Secondary Total Treatment Effect (STTE) on ineligible units. Under randomization within the eligible set, we give identification conditions and develop interference-aware ensemble estimators that combine exposure mappings, generalized propensity scores, and flexible machine learning. We further introduce a projection that links treatment- and outcome-level estimands; this mapping is exact under a Linear Additive Edges condition and enables estimation on the (typically much smaller) treatment side with deterministic aggregation to outcomes. In simulations with known ground truth across realistic exposure regimes, the proposed estimators recover PTTE and STTE with low bias and variance and reduce the bias that could arise when interference is ignored. Two field experiments illustrate practical relevance: our method corrects the direction of expected interference bias for a pre-specified metric in both studies and reverses the sign and significance of the primary decision metric in one case.

Estimating Total Effects in Bipartite Experiments with Spillovers and Partial Eligibility

TL;DR

This work addresses estimation of total effects in eligibility-constrained bipartite experiments where only a subset of treatment-side units can be randomized, but interference propagates across all units. It introduces Primary Total Treatment Effect (PTTE) and Secondary Total Treatment Effect (STTE) and develops ensemble estimators that fuse exposure mappings, generalized propensity scores, and machine learning, with a projection linking treatment- and outcome-level estimands under a Linear Additive Edges assumption. The projection enables efficient estimation at the treatment level with aggregation to outcomes, yielding accurate PTTE and STTE estimates in simulations and two real-field experiments, and highlights how ignoring spillovers can bias decision metrics. The framework provides practical guidance for deploying interference-aware evaluations in bipartite systems and points to directions for handling partially observed networks, dynamic settings, and non-additive outcomes, with significant implications for rollout planning in platform economies.

Abstract

We study randomized experiments in bipartite systems where only a subset of treatment-side units are eligible for assignment while all units continue to interact, generating interference. We formalize eligibility-constrained bipartite experiments and define estimands aligned with full deployment: the Primary Total Treatment Effect (PTTE) on eligible units and the Secondary Total Treatment Effect (STTE) on ineligible units. Under randomization within the eligible set, we give identification conditions and develop interference-aware ensemble estimators that combine exposure mappings, generalized propensity scores, and flexible machine learning. We further introduce a projection that links treatment- and outcome-level estimands; this mapping is exact under a Linear Additive Edges condition and enables estimation on the (typically much smaller) treatment side with deterministic aggregation to outcomes. In simulations with known ground truth across realistic exposure regimes, the proposed estimators recover PTTE and STTE with low bias and variance and reduce the bias that could arise when interference is ignored. Two field experiments illustrate practical relevance: our method corrects the direction of expected interference bias for a pre-specified metric in both studies and reverses the sign and significance of the primary decision metric in one case.

Paper Structure

This paper contains 33 sections, 1 theorem, 26 equations, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Intuition Behind Primary and Secondary Treatment Effects
  3. Related Work
  4. Methodology
  5. Problem setup
  6. Edge-level representation and additive metrics.
  7. Primary and secondary outcome components.
  8. Outcome-Unit Ensemble
  9. Treatment-Unit Ensemble
  10. Formal Assumptions
  11. Projection Between Treatment and Outcome Effects
  12. Estimation Methods
  13. Step 1: Feature Construction.
  14. Step 2: Outcome Function Estimation.
  15. Step 3: Counterfactual Prediction and Aggregation.
  16. ...and 18 more sections

Key Result

Theorem 2.5

Suppose Assumption ass:linear-additive holds. Then where $\text{PTTE}_{\text{outcome}}$ is defined in eq:PTTE-outcome-primary in terms of $Y_{i,\text{prim}}(\mathbf Z)$ and $\text{PTTE}_{\text{treatment}}$ is defined in eq:PTTE-treatment in terms of $Y_j(\mathbf Z)$.

Figures (6)

  • Figure 1: Bipartite structure in eligibility-constrained experiments. Primary units $\mathcal{T}_{\text{prim}}$ (top left) are eligible for treatment; secondary units $\mathcal{T}_{\text{sec}}$ (bottom left) are ineligible but connected to outcomes. Outcome units $\mathcal{O}_{\text{prim}}$ (top right) connect to primaries; $\mathcal{O}_{\text{Both}}$ (overlap) connect to both.
  • Figure 2: PTTE estimates for outcome units (left) and treatment units (right), showing ground truth (GT), Basic, linear polynomial (LP), Kernel Ridge Regression (KRR), and its projected variant (Proj. KRR, outcome level only).
  • Figure 3: Box plot of PTTE at the outcome unit granularity
  • Figure 4: Box plot of PTTE at the treatment unit granularity
  • Figure 5: Box plot of STTE at the outcome unit granularity
  • ...and 1 more figures

Theorems & Definitions (5)

  • Theorem 2.5: Exact projection of PTTE under edge additivity
  • proof
  • Remark 2.6
  • Remark 2.7
  • Remark 2.8