Personalization Aids Pluralistic Alignment Under Competition

TL;DR

This work shows that under a Weak Market Alignment condition, every equilibrium gives each user outcomes comparable to those from a perfectly aligned common model -- so personalization can induce pluralistically aligned outcomes, even when providers are self-interested.

Abstract

Can competition among misaligned AI providers yield aligned outcomes for a diverse population of users, and what role does model personalization play? We study a setting where multiple competing AI providers interact with multiple users who must make downstream decisions but differ in preferences. Providers have their own objectives over users' actions and strategically deploy AI models to advance them. We model the interaction as a Stackelberg game with multiple leaders (providers) and followers (users): providers commit to conversational policies, and users choose which model to use, how to converse, and how to act. With user-specific personalization, we show that under a Weak Market Alignment condition, every equilibrium gives each user outcomes comparable to those from a perfectly aligned common model -- so personalization can induce pluralistically aligned outcomes, even when providers are self-interested. In contrast, when providers must deploy a single anonymous policy, there exist equilibria with uninformative behavior under the same condition. We then give a stronger alignment condition that guarantees each user their optimal utility in the anonymous setting.

Figures (10)

• Figure 1: (Top) Weak Market Alignment improves with more providers. Mean root mean squared error (RMSE, across groups) vs. number of available providers $K$. Nonnegative least squares (NNLS)+intercept outperforms baselines; error decreases with $K$. (Bottom) Some groups are harder to align. Per-group RMSE vs. $K$. Within each partition, alignment quality varies across demographic groups. Shaded bands: CV standard error.
• Figure 2: (Top) More providers reduce transfer factors. Transfer factor $1/\lambda_i^*$ vs. number of providers $K$: mean over users (solid) and worst user (dashed). The dramatic drop shows how provider diversity improves coverage. (Bottom) More users: lower error but harder coverage. Worst-user transfer factor $1/\lambda_i^*$ (red, left axis) vs. provider fitting error ${\varepsilon}$ (blue, right axis) as number of users increases.
• Figure 3: Not all providers need to be aligned. Worst-user transfer factor $1/\lambda_i^*$ vs. size of aligned subset $|T|$ (log scale). Mean $T$ (blue) and worst $T$ (red) show that a random small subset often fails; the best subset (green) achieves low transfer even at $|T|=1$. Insets zoom to linear scale, revealing the gap between best and mean/worst at large $|T|$.
• Figure 4: Scoring rule robustness. (a,b) Weak alignment RMSE under log and Brier scores (Political Ideology). (c,d) Transfer factors across all three scores. Patterns are stable.
• Figure 5: Weak alignment across demographics. Mean RMSE vs. $K$ for eight additional partitions. NNLS outperforms baselines in all cases.

Theorems & Definitions (31)

• Definition 2.1: Player Strategies
• Definition 2.2: The Personalized Game
• Definition 2.3: The Anonymous Game
• Definition 2.4: Induced distribution $\mathcal{I}_i(C_{P}, C_{U},D_{U};j)$
• Definition 2.5: User's Best Response
• Definition 2.6: Nash Equilibrium in the Personalized Game
• Definition 2.7: Nash Equilibrium in the Anonymous Game
• Definition 2.8: Common garbling
• Definition 2.9: $T$-Shared Conversation Rules $\mathcal{C}_{S}(x^{P}_{T})$
• Definition 2.10: User $i$'s utility when interacting with provider $j$ who deploys $C_{P_j}$