Statistical Inference under Performativity

TL;DR

This work tackles statistical inference when predictions influence the data they predict (performativity) by developing an end-to-end framework for inference under performativity. It establishes a central limit theorem for estimators produced by repeated risk minimization and extends prediction-powered inference to dynamic performative settings, including data-driven covariance estimation via score matching and a policy-perturbation scheme. The framework enables bias-aware confidence regions for the performative stable point and tighter inference through PPI, with a greedy method to select weighting parameters that improve efficiency. Through simulations and a semi-synthetic credit-scoring case, the approach yields robust coverage and narrower confidence intervals, demonstrating practical value for policy design and economic decision-making under shifting data-generating processes.

Abstract

Performativity of predictions refers to the phenomenon where prediction-informed decisions influence the very targets they aim to predict -- a dynamic commonly observed in policy-making, social sciences, and economics. In this paper, we initiate an end-to-end framework of statistical inference under performativity. Our contributions are twofold. First, we establish a central limit theorem for estimation and inference in the performative setting, enabling standard inferential tasks such as constructing confidence intervals and conducting hypothesis tests in policy-making contexts. Second, we leverage this central limit theorem to study prediction-powered inference (PPI) under performativity. This approach yields more precise estimates and tighter confidence regions for the model parameters (i.e., policies) of interest in performative prediction. We validate the effectiveness of our framework through numerical experiments. To the best of our knowledge, this is the first work to establish a complete statistical inference under performativity, introducing new challenges and inference settings that we believe will provide substantial value to policy-making, statistics, and machine learning.

Figures (7)

• Figure 1: Confidence-region coverage (top row) and width (bottom row) with different choices of $\lambda$. The left, middle, and right columns correspond to inference steps $t=2$, $t=3$, and $t=4$, respectively. The solid and dashed curves correspond to the confidence-region coverage for $\theta_t$ and $\theta_{\rm PS}$, respectively.
• Figure 2: Visualizations to verify the Central Limit Theorem. (a) plots the density map of sampled $\widehat{V}_t^{-1/2}\sqrt{n}(\widehat{\theta}_t - \theta_t)$, while (b) compares the observed distribution with theoretical one $\mathcal{N}(0,I_d)$.
• Figure 3: Evaluating the estimation quality of two designed score matching models.
• Figure A1: Confidence-region coverage (top row) and width (bottom row) with different choices of $\lambda$. The setup is the same as in Figure \ref{['fig:CI_1']}, only we change $\gamma =1$ or $\gamma =3$.
• Figure A2: Confidence-region coverage (top row) and width (bottom row) with different choices of $\lambda$. The setup is the same as in Figure \ref{['fig:CI_1']}, only we change $\varepsilon\approx 0.003$ or $\varepsilon\approx 0.03$.

Theorems & Definitions (26)

• Theorem 2.1: Informal, adopted from perdomo2020performative
• Remark 3.2
• Proposition 3.3
• Theorem 3.4: Central Limit Theorem of $\widehat{\theta}_t$
• Lemma 3.5
• Theorem 3.6
• Corollary 3.7: Confidence region construction for $\theta_{\text{PS}}$
• Theorem 4.1: Central Limit Theorem of $\widehat{\theta}^{\text{PPI}}_t(\lambda_t)$
• Corollary 4.2
• Proposition A.6: Consistency of $\widehat{\theta}_t$, Restatement of Proposition \ref{['prop:consistency']}