Decision-Dependent Stochastic Optimization: The Role of Distribution Dynamics

TL;DR

This paper develops an online algorithm that achieves optimal decision-making by both adapting to and shaping the dynamic distribution and showcases the theoretical results in an opinion dynamics context, where an opportunistic party maximizes the affinity of a dynamic polarized population and in a recommender system scenario, featuring performance optimization with discrete distributions in the probability simplex.

Abstract

Distribution shifts have long been regarded as troublesome external forces that a decision-maker should either counteract or conform to. An intriguing feedback phenomenon termed decision dependence arises when the deployed decision affects the environment and alters the data-generating distribution. In the realm of performative prediction, this is encoded by distribution maps parameterized by decisions due to strategic behaviors. In contrast, we formalize an endogenous distribution shift as a feedback process featuring nonlinear dynamics that couple the evolving distribution with the decision. Stochastic optimization in this dynamic regime provides a fertile ground to examine the various roles played by dynamics in the composite problem structure. To this end, we develop an online algorithm that achieves optimal decision-making by both adapting to and shaping the dynamic distribution. Throughout the paper, we adopt a distributional perspective and demonstrate how this view facilitates characterizations of distribution dynamics and the optimality and generalization performance of the proposed algorithm. We showcase the theoretical results in an opinion dynamics context, where an opportunistic party maximizes the affinity of a dynamic polarized population, and in a recommender system scenario, featuring performance optimization with discrete distributions in the probability simplex.

Figures (8)

• Figure 1: Stochastic optimization with decision dependence features a closed loop, involving endogenous distribution shifts from $\mu_k$ to $\mu_{k+1}$ due to the decision $u$ and dynamics. Here $\Phi$ is an objective function of the decision $u$ and the random variable $p$, see problem \ref{['eq:dd_opt_problem']} in \ref{['subsec:prob_formulation']} for a formal account.
• Figure 2: The polarized dynamics \ref{['eq:polarized_dynamics']} admit a unique steady state depending on the sign of $p_0^{\top} q$.
• Figure 3: This figure illustrates the convergence behaviors of our online stochastic algorithm \ref{['eq:stochastic_composite']} (termed "composite") and the vanilla algorithm \ref{['eq:stochastic_vanilla']} (termed "vanilla") oblivious of the composite structure due to decision dependence. The solid lines represent the average values of convergence measures, whereas the shaded areas indicate the ranges of change in various independent trials.
• Figure 4: This figure illustrates the histograms of the angles between the position and the decision across the population. Initially, the angles are largely concentrated in $[80^{\circ}, 100^{\circ}]$, implying the average population-wide affinity is close to zero. In the end (i.e., when the population reaches the steady state), these angles mostly fall into $[40^\circ, 80^\circ]$, which indicates that the average affinity becomes positive.
• Figure 5: This figure illustrates the convergence behaviors of the algorithm \ref{['eq:stochastic_vanilla_simplex']} (labeled "vanilla") unaware of the composite problem structure, the derivative-free algorithm \ref{['eq:stochastic_dfo_simplex']} (labeled "derivative-free"), and our online stochastic algorithm \ref{['eq:stochastic_composite_simplex']} (labeled "composite").

Theorems & Definitions (45)

• Example 1: Ideology tailored to a polarized population
• Example 2: Linear distribution dynamics
• Lemma 1
• Lemma 2
• Theorem 3
• Lemma 4
• Theorem 5
• Corollary 6
• Theorem 7
• Theorem 8