Fair-PP: A Synthetic Dataset for Aligning LLM with Personalized Preferences of Social Equity

TL;DR

This work tackles the challenge of aligning LLMs with personalized social equity preferences by introducing Fair-PP, a synthetic dataset built from real-world survey data that spans 28 social groups, 98 topics, and 5 value-dimension perspectives, yielding 34,089 questions and 238,623 personalized preference records anchored by seven persona profiles. It provides an automated data-generation framework and analyzes how mainstream LLMs position themselves in the resulting personalized preference space across regions, complemented by a sample weighting method to align models to a target persona while maximizing divergence from others. Three experiments demonstrate that weighting-based alignment methods outperform baselines and generalize to simulated scenarios, highlighting WDPO and WSFT as effective for persona-specific alignment. The dataset and methods offer a practical pathway toward multi-persona, culturally aware AI that better reflects diverse societal values in governance and public-service contexts.

Abstract

Human preference plays a crucial role in the refinement of large language models (LLMs). However, collecting human preference feedback is costly and most existing datasets neglect the correlation between personalization and preferences. To address this issue, we introduce Fair-PP, a synthetic dataset of personalized preferences targeting social equity, derived from real-world social survey data, which includes 28 social groups, 98 equity topics, and 5 personal preference dimensions. Leveraging GPT-4o-mini, we engage in role-playing based on seven representative persona portrayals guided by existing social survey data, yielding a total of 238,623 preference records. Through Fair-PP, we also contribute (i) An automated framework for generating preference data, along with a more fine-grained dataset of personalized preferences; (ii) analysis of the positioning of the existing mainstream LLMs across five major global regions within the personalized preference space; and (iii) a sample reweighting method for personalized preference alignment, enabling alignment with a target persona while maximizing the divergence from other personas. Empirical experiments show our method outperforms the baselines.

Figures (8)

• Figure 1: Landscape of space.
• Figure 2: An overview of the dataset. Each question consists of three parts: the social group, an equity topic, and a perspective dimension. An example question is shown on the right where option A and B represent two types of viewpoints under a specific dimension. Personalized preferences are collected through LLM personalization that leverages 7 value portrait based on the real-world social surveys.
• Figure 3: Personalized preference anchors. Blue and red represent the proportions of choices for option A and option B, respectively.
• Figure 4: Fine-grained Personalized Preferences and Aggregate Distribution Across Personas, Social Groups, and Equity Topics: The scatter plot shows the proportion of option A selected for each persona across the combined social group and equity topic categories (with point size scaled by the proportion). The bar plots on the left and top show the overall option distribution for each equity topic and social group, respectively.
• Figure 5: Personalized preference analysis: (a) we calculate the Jensen-Shannon similarity between seven personas. (b) Targeting Persona 6 (Abbreviated as P6), we analyze preference matching frequency with other personas and quantify the number of questions in each distinct frequency tier, then (c) For persona 6's unique preferences, we list the top-5 most frequent social groups and equity topics.