Performative Prediction with Bandit Feedback: Learning through Reparameterization

TL;DR

This work tackles performative prediction with bandit feedback by introducing a reparameterization that recasts the non-convex performative risk $\mathsf{PR}(\theta)$ as a convex function $\mathsf{PR}^\dagger(\phi)$ over the induced distribution parameter $\phi=\varphi(\theta)$. It then proposes a two-level zeroth-order optimization framework: an outer loop optimizes $\mathsf{PR}^\dagger(\phi)$ in $\phi$, while an inner LearnModel subroutine finds a model $\theta$ that induces a target distribution parameter $\phi$, using KL-divergence based objectives and KL-oracle access. Under mild Lipschitz and convexity assumptions, the authors establish sublinear regret bounds in the total number of samples, specifically $\widetilde{O}((d_\Theta+d_\Phi) N_{\mathsf{KL}}^{1/6} N^{5/6})$, implying convergence to the performative optimum. The approach enables gradient-free learning, robustness to noise, and applicability to black-box environments, with empirical validation on a toy example showing faster convergence than baseline methods. Overall, the work advances scalable, gradient-free strategies for performative prediction in settings with unknown distribution maps and bandit feedback, suggesting practical impact for dynamic, feedback-driven predictive systems.

Abstract

Performative prediction, as introduced by Perdomo et al, is a framework for studying social prediction in which the data distribution itself changes in response to the deployment of a model. Existing work in this field usually hinges on three assumptions that are easily violated in practice: that the performative risk is convex over the deployed model, that the mapping from the model to the data distribution is known to the model designer in advance, and the first-order information of the performative risk is available. In this paper, we initiate the study of performative prediction problems that do not require these assumptions. Specifically, we develop a reparameterization framework that reparametrizes the performative prediction objective as a function of the induced data distribution. We then develop a two-level zeroth-order optimization procedure, where the first level performs iterative optimization on the distribution parameter space, and the second level learns the model that induces a particular target distribution at each iteration. Under mild conditions, this reparameterization allows us to transform the non-convex objective into a convex one and achieve provable regret guarantees. In particular, we provide a regret bound that is sublinear in the total number of performative samples taken and is only polynomial in the dimension of the model parameter.

Figures (4)

• Figure 1: Illustration of our procedure (\ref{['algorithm:minimize-indirectly-convex-function']}). Each big block represents one iteration of the outer algorithm, which consists of three sub-steps: Step 1, the learner first computes the two target distribution $\phi^{+}_t$ and $\phi^{-}_t$ (corresponds to the white section), Step 2, the learner uses $\operatorname{\mathsf{LearnModel}}$ to learn the corresponding model $\hat{\theta}^{+}_t$ and $\hat{\theta}^{-}_t$ that can best approximately induce $\phi^{+}_t$ and $\phi^{-}_t$(corresponds to the grey section) correspondingly. Step 3, the learner deploys $\hat{\theta}^{+}_t$ and $\hat{\theta}^{-}_t$ and perform a gradient update and get $\phi_{t+1}$. Each deployment of $\operatorname{\mathsf{LearnModel}}$ requires a total number of $S$ steps. Thus, the total number of steps involved in the whole procedure is $T_{\textsf{total}} = T\times S$.
• Figure 2: An example showing that our assumption is weaker than the mixture dominance assumption in miller2021outside. In the left figure, the blue curve represents the function $\mathsf{PR}^\dagger(\varphi_\theta)$ which is convex w.r.t the data distribution parameter $\varphi_\theta$; while the red curve represents the function $\mathsf{PR}(\theta)$, which is not a convex function with respect to $\theta$. In the right two figures, we compare $\mathsf{PR}$ as a function of the model parameter $\theta$ and as a function of the distribution parameter $\phi$.
• Figure 3: Another example showing $\mathsf{PR}$ is convex in $\phi$ but not $\theta$. The original $\mathsf{PR}$ loss $\mathsf{PR}(\theta)$ is in red, which is non-convex), and the reformulated PR loss $\mathsf{PR}^\dagger (\theta)$ is in blue via reparameterization, which is convex)
• Figure 4: Empirical results comparing baseline method (zeroth-order optimization without reparametrization, orange curve) vs. our method (zeroth order optimization after reparametrization, (blue curve) based on \ref{['example:mixture-dominance-too-strong-condition']}.

Theorems & Definitions (52)

• Lemma 1
• Proposition 1: Sublinear Regret Implies Convergence
• Example 1
• Example 2
• Example 3
• Example 4: Gaussian distribution
• Example 5: Uniform distribution
• Remark 1
• Theorem 1: High-probability regret bound for \ref{['algorithm:minimize-indirectly-convex-function']} in $T$
• Example 1