Qini Curve Estimation under Clustered Network Interference

TL;DR

Addressing Qini curve estimation under clustered network interference, the paper demonstrates that standard no-interference methods yield biased policy assessment. It provides identifiability results for policy value $V(oldsymbol{ ext{pi}})$ and policy cost $C(oldsymbol{ ext{pi}})$ and proposes three estimation strategies—cluster-level IPW, fractional IPW under a fractional exposure mapping, and $eta$-IPW under a beta-additive interference model—along with a marketplace simulator. Through simulations, it shows that ignoring interference biases Qini curves, while the proposed methods achieve favorable bias-variance trade-offs, with $eta$-IPW often delivering strong MSE performance and ranking accuracy. The work offers practical guidance for policy evaluation under networked interference and provides a reusable simulator for future research.

Abstract

Qini curves are a widely used tool for assessing treatment policies under allocation constraints as they visualize the incremental gain of a new treatment policy versus the cost of its implementation. Standard Qini curve estimation assumes no interference between units: that is, that treating one unit does not influence the outcome of any other unit. In many real-life applications such as public policy or marketing, however, the presence of interference is common. Ignoring interference in these scenarios can lead to systematically biased Qini curves that over- or under-estimate a treatment policy's cost-effectiveness. In this paper, we address the problem of Qini curve estimation under clustered network interference, where interfering units form independent clusters. We propose a formal description of the problem setting with an experimental study design under which we can account for clustered network interference. Within this framework, we describe three estimation strategies, each suited to different conditions, and provide guidance for selecting the most appropriate approach by highlighting the inherent bias-variance trade-offs. To complement our theoretical analysis, we introduce a marketplace simulator that replicates clustered network interference in a typical e-commerce environment, allowing us to evaluate and compare the proposed strategies in practice.

Figures (4)

• Figure 1: An illustrative simulation study with Qini curve estimation under clustered network interference. The black dashed line represents the true underlying Qini curve, while the solid lines depict two estimation approaches: one based on a traditional method that assumes no interference, and the other representing a proposed strategy in this paper that adjusts for interference using inverse probability weighting. More details on the simulation used to generate this figure can be found in Appendix \ref{['app:experiments']}.
• Figure 2: Comparison of bias and mean squared error for each strategy under different interference structures. We let $N=100.000$ and $M=11$. Averages and standard errors are reported over 150 repetitions.
• Figure 3: Qini curves of each estimation strategy with $N=100.000$ and $M=11$ using with the exponential decay function $\eta_{\text{exp-decay}}$. The average Qini curve and standard error are reported over 150 repetitions. The dashed black line corresponds to the true underlying Qini curve.
• Figure 4: Comparing variance of each estimation strategy as we vary the number the buyers $N$ (i.e., clusters) or items $M$ (i.e., units) with $\eta_{\text{exp-decay}}$. While varying one, the other is kept fixed to either $N=20.000$ or $K=11$. The variance is reported over 150 repetitions.

Theorems & Definitions (11)

• Lemma 1
• Theorem 1
• Theorem 2
• Theorem 3
• Lemma 2
• proof
• proof
• proof
• proof
• Theorem 4