GRAVITY: A Framework for Personalized Text Generation via Profile-Grounded Synthetic Preferences

TL;DR

GRAVITY presents a scalable framework for personalized text generation by synthesizing profile-grounded preferences from demographic, cultural, and psychological signals. It constructs structured user profiles, generates scenario-based preferences across interests, values, and personality, and fine-tunes LLMs with Direct Preference Optimization to produce user-aligned book descriptions. Across 400 diverse readers and multiple cultures, GRAVITY yields consistent improvements over baselines and is preferred by users in user studies, illustrating the value of scenario-driven synthetic data. The approach reduces annotation needs while capturing richer user variation, enabling scalable, interpretable personalization in LLMs.

Abstract

Personalization in LLMs often relies on costly human feedback or interaction logs, limiting scalability and neglecting deeper user attributes. To reduce the reliance on human annotations, we introduce GRAVITY (Generative Response with Aligned Values, Interests, and Traits of You), a framework for generating synthetic, profile-grounded preference data that captures users' interests, values, beliefs, and personality traits. By integrating demographic, cultural, and psychological frameworks -- including Hofstede's cultural dimensions, Schwartz's basic values, the World Values Survey, and Big Five OCEAN traits -- GRAVITY synthesizes preference pairs to guide personalized content generation. We evaluate GRAVITY on book descriptions for 400 Amazon users, comparing it to prompt-based conditioning, standard fine-tuning, and naive synthetic pair generation. Profile-grounded synthetic data consistently improves generation, especially across multiple cultures (USA, Brazil, Japan, India), achieving over 4% higher preference gains across baselines, with user studies showing that GRAVITY outputs are preferred over 86% of the time. Our results show that scenario-grounded synthetic data can capture richer user variation, reduce reliance on costly annotation, and produce more engaging, user-centered content, offering a scalable path for LLM personalization.

Figures (6)

• Figure 1: Our approach GRAVITY consists of four stages: (1) User Profile Creation, (2) Custom Preference Data Selection, (3) User-Centric LLM Training, and (4) Product Description Personalization. In stage 1, we extract information about the user including explicit values such as demographic attributes: age, location, and gender and users' interests and implicit values including their personality traits and their values and beliefs (extracted based on seed statements generated from various psychological and cultural frameworks). In stage 2, During stage 1, we generate a candidate pool of scenarios for values and beliefs based on Stage 1 using GPT-4o. We then generate a custom set of chosen/reject preference pairs for users spanning three facets: user interests, personality, and values using a combination of the Amazon Reviews dataset, personality SJTs (TRAIT and Big5Chat), and the candidate pool of generated scenarios. In stage 3, we preference tune Llama with the users' preference pairs, and finally, in Stage 4, we generate the personalized description by prompting this tuned model with user profile attributes.
• Figure 2: Preference Gains (%) for personalized generation methods across users from four diverse countries: USA, Brazil, Japan, and India. We observe that GRAVITY yields consistently strong personalization metrics across the cultures, with gains in non-Western countries compared to current approaches (>10% increase in preference gains).
• Figure 3: Preference gains (%) across nine book genres (similar categories, clustered into fiction and non-fiction) using GRAVITY. We find that most fiction books have much higher preference gains (> 70%) as compared to non-fiction books. ($\approx$ 65%).
• Figure 4: Example questions from first stage in user study: data collection. Along with standard demographics, we extract user interests i.e. ranking top 3 genres, value systems, and personality.
• Figure 5: Example questions from second stage in user study: user values verification. Users are given additional values seed statements and asked to verify whether they agree with GPT-4o annotations.