Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TEA-Time: Transporting Effects Across Time

Harsh Parikh, Gabriel Levin-Konigsberg, Dominique Perrault-Joncas, Alexander Volfovsky

Abstract

Treatment effects estimated from randomized controlled trials are local not only to the study population but also to the time at which the trial was conducted. We develop a framework for temporal transportation: extrapolating treatment effects to time periods where no experiment was conducted. We target the transported average treatment effect (TATE) and show that under a separable temporal effects assumption, the TATE decomposes into an observed average treatment effect and a temporal ratio. We provide two identification strategies -- one using replicated trials comparing the same treatments at different times, another using common treatment arms observed across time -- and develop doubly robust, semiparametrically efficient estimators for each. Monte Carlo simulations confirm that both estimators achieve nominal coverage, with the common arm strategy yielding substantial efficiency gains when its stronger assumptions hold. We apply our methods to A/B tests from the Upworthy Research Archive, demonstrating that the two strategies exhibit a variance-bias tradeoff: the common arm approach offers greater precision but may incur bias when treatments interact heterogeneously with temporal factors.

TEA-Time: Transporting Effects Across Time

Abstract

Treatment effects estimated from randomized controlled trials are local not only to the study population but also to the time at which the trial was conducted. We develop a framework for temporal transportation: extrapolating treatment effects to time periods where no experiment was conducted. We target the transported average treatment effect (TATE) and show that under a separable temporal effects assumption, the TATE decomposes into an observed average treatment effect and a temporal ratio. We provide two identification strategies -- one using replicated trials comparing the same treatments at different times, another using common treatment arms observed across time -- and develop doubly robust, semiparametrically efficient estimators for each. Monte Carlo simulations confirm that both estimators achieve nominal coverage, with the common arm strategy yielding substantial efficiency gains when its stronger assumptions hold. We apply our methods to A/B tests from the Upworthy Research Archive, demonstrating that the two strategies exhibit a variance-bias tradeoff: the common arm approach offers greater precision but may incur bias when treatments interact heterogeneously with temporal factors.
Paper Structure (88 sections, 11 theorems, 85 equations, 3 figures, 3 tables)

This paper contains 88 sections, 11 theorems, 85 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Setup and Assumptions
  4. Notation.
  5. Standard assumptions.
  6. Separable temporal effects.
  7. Target estimand.
  8. Identification
  9. TATE Decomposition
  10. Strategy 1: Replicated Trials
  11. Strategy 2: Common Arm
  12. Comparing the Two Strategies
  13. Estimation and Inference
  14. Efficient Influence Functions
  15. Building blocks.
  16. ...and 73 more sections

Key Result

Theorem 1

Under random assignment, SUTVA, and Assumption ass:separable: where $\tau_{k^\star}(0,0) = \mathbb{E}[Y_{t_{1k^\star}} \mid A = a_{k^\star}, S = k^\star] - \mathbb{E}[Y_{t_{1k^\star}} \mid A = b_{k^\star}, S = k^\star]$ is the observed ATE.

Figures (3)

  • Figure 1: Temporal transportation uses anchor trials to extrapolate treatment effects across time. Information flows from the primary trial $k^\star$ (blue) to contemporary anchor trials (orange, green) at the source time, then across time via anchors observed at multiple periods (dashed arrows), and finally from target-time anchors to the transported effect (TATE, dashed blue). This chain identifies how effects scale temporally without requiring the primary comparison at the target time.
  • Figure 2: TATE estimates for Trial A (Cluster 17 vs. 10) by target month. Strategy 1 (blue) exhibits higher variance but tracks the true TATE (green); Strategy 2 (orange) is more precise but shows systematic bias. Shaded regions indicate 95% CIs.
  • Figure 3: TATE estimates by target month for both trials. Each panel shows Strategy 1 (Replicated Trials; blue circles), Strategy 2 (Common Arm; orange squares), and true TATE (green diamonds) with 95% confidence bands. Dashed red line indicates observed ATE at source time. Left: Trial A (Cluster 17 vs. 10), source time January 2014. Right: Trial B (Cluster 46 vs. 12), source time December 2013.

Theorems & Definitions (24)

  • Theorem 1: TATE Decomposition
  • Theorem 2: Identification via Replicated Trials
  • Theorem 3: Identification via Common Arm
  • Proposition 1: Efficient Influence Functions
  • Theorem 4: Asymptotic Properties
  • proof : Proof of Theorem \ref{['thm:decomposition']}
  • proof : Proof of Theorem \ref{['thm:replicated']}
  • proof : Proof of Theorem \ref{['thm:common_arm']}
  • Lemma 1: Properties of Doubly Robust Scores
  • proof
  • ...and 14 more