Unbiased Platform-Level Causal Estimation for Search Systems: A Competitive Isolation PSM-DID Framework

TL;DR

This work tackles the challenge of estimating platform-level causal effects in search-based two-sided marketplaces under widespread interference. It presents the Competitive Isolation PSM-DID (CI-PSM-DID) framework, which fuses min-cut mutual-exclusion graph partitioning with Stratified CTCVR Matching and a two-sided sinking DID to yield unbiased estimates that align with perfect A/B testing under the assumptions of mutual exclusion and parallel trends, i.e., $\hat{\tau} \equiv \Delta^{*}$. The authors provide theoretical guarantees, scalable algorithms, and extensive offline and online validation, achieving substantial reductions in cannibalization and estimation variance, and demonstrating actionable platform-level lift measurements (e.g., GMV and order volume) at scale. An open dataset is released to support reproducible research on marketplace interference, enhancing transparency and transferability of the framework. Overall, CI-PSM-DID offers a practical, scalable tool for robust platform-level causal inference in marketplaces where cross-unit interference is pervasive.

Abstract

Evaluating platform-level interventions in search-based two-sided marketplaces is fundamentally challenged by systemic effects such as spillovers and network interference. While widely used for causal inference, the PSM (Propensity Score Matching) - DID (Difference-in-Differences) framework remains susceptible to selection bias and cross-unit interference from unaccounted spillovers. In this paper, we introduced Competitive Isolation PSM-DID, a novel causal framework that integrates propensity score matching with competitive isolation to enable platform-level effect measurement (e.g., order volume, GMV) instead of item-level metrics in search systems. Our approach provides theoretically guaranteed unbiased estimation under mutual exclusion conditions, with an open dataset released to support reproducible research on marketplace interference (github.com/xxxx). Extensive experiments demonstrate significant reductions in interference effects and estimation variance compared to baseline methods. Successful deployment in a large-scale marketplace confirms the framework's practical utility for platform-level causal inference.

Figures (4)

• Figure 1: Competition effects between item sets: When $A$ is introduced alongside $C$, it competes for shared demand (shaded area). The competition effect $\Gamma_A(C)|_{T}^{AC} = \mathbb{E}[C|_{\overline{A}, T}] - \mathbb{E}[(A + C)_{T}^{AC}] + \mathbb{E}[A_{T}^{AC}]$ quantifies $C$'s metric loss due to $A$'s presence.
• Figure 2: (a) Perfect A/B. The same item receives different treatments across buckets; if the treatment is a price change, “same-item different-price” creates public-sentiment/compliance risk and is infeasible. (b) CI-PSM-DID. A mutual-exclusion graph partition (§\ref{['subsec:subgraph_partition']}) enforces $\Gamma_B(A)=\Gamma_A(B)=0$, so $A$'s treatment $A\!\to\!A'$ does not affect $B$. Via PSM, $B$ is chosen homogeneous to $A$ to satisfy parallel trends at $T_0$, hence $B{+}C$ proxies pre-treatment $A{+}C$. Measurement uses two-sided sinking: control observes $Y(B{+}C \,\|_{\overline{A}}, T_t)$, treatment observes $Y(A'{+}C \,\|_{\overline{B}}, T_t)$; the DID over $T_0\!\to\!T_1$ identifies the platform-level effect.
• Figure 3: Competitive Isolation PSM-DID Framework Schematic. (T0): Pre-intervention state with two isolated buckets: (1) Control group sinks $A$ to observe $B + C$ (metric: $Y(B + C \|_{\overline{A}})$), (2) Treatment group sinks $B$ to observe $A + C$ (metric: $Y(A + C \|_{\overline{B}})$). The difference $D_0 = Y(A + C \|_{\overline{B}}) - Y(B + C \|_{\overline{A}})$ establishes the baseline. (T1): Post-intervention: $A$ receives treatment (becoming $A'$), while sinking configuration remains identical. $B$ does not receive treatment and, due to mutual exclusion, $A$'s treatment has no effect on $B$. The new difference $D_1 = Y(A' + C \|_{\overline{B}}) - Y(B + C \|_{\overline{A'}})$ captures treatment effects. The causal estimate $\hat{\tau} = D_1 - D_0$ eliminates temporal confounders and interference biases.
• Figure 4: Offline GMV gap comparison (7-day, repeat 30 times) across sample sizes: Our method (CI-PSM-DID-Stratified CTCVR Matching) achieves the lowest gaps at 150K-600K scales.