Text as a Universal Interface for Transferable Personalization

TL;DR

The paper tackles the lack of transferability and interpretability in traditional user representations for personalization in large language models by introducing a text-based universal interface of textual preference summaries. It presents AlignXplore+ and a two-stage training framework: supervised fine-tuning using a generate-validate-merge pipeline to produce high-quality summaries, followed by reinforcement learning with curriculum-based data pruning and a cumulative long-horizon reward to optimize for streaming updates. Across nine benchmarks spanning recommendation, response selection, and response generation, an ~8B model achieves state-of-the-art performance and strong zero-shot transferability across domains and downstream models, outperforming substantially larger open-source models. The approach enhances interpretability, enables cross-domain and cross-model personalization, and supports streaming updating of user preferences, representing a practical step toward universal, user-centric AI interfaces.

Abstract

We study the problem of personalization in large language models (LLMs). Prior work predominantly represents user preferences as implicit, model-specific vectors or parameters, yielding opaque ``black-box'' profiles that are difficult to interpret and transfer across models and tasks. In contrast, we advocate natural language as a universal, model- and task-agnostic interface for preference representation. The formulation leads to interpretable and reusable preference descriptions, while naturally supporting continual evolution as new interactions are observed. To learn such representations, we introduce a two-stage training framework that combines supervised fine-tuning on high-quality synthesized data with reinforcement learning to optimize long-term utility and cross-task transferability. Based on this framework, we develop AlignXplore+, a universal preference reasoning model that generates textual preference summaries. Experiments on nine benchmarks show that our 8B model achieves state-of-the-art performanc -- outperforming substantially larger open-source models -- while exhibiting strong transferability across tasks, model families, and interaction formats.

Figures (4)

• Figure 1: Paradigm shift from vector/parameter-based (a) to text-based user representations (b) for personalization. (a) Traditional methods generate user-specific parameters and vectors that are tightly coupled with their training model, thus cannot be transferred. (b) We pioneer a text-based paradigm that infers a model- and task-agnostic preference summary, which serves as a universal interface to empower any downstream model for various tasks.
• Figure 2: Overview of our training framework, which consists of two main stages. (a) SFT Stage: We first create high-quality training data using a "generate-validate-merge" pipeline, which synthesizes a comprehensive preference summary $P^k$ by ensuring accurate prediction of multiple future interactions. (b) & (c) RL Stage: This stage features a curriculum pruning strategy that uses select reasoning-intensive samples, and a cumulative reward function that optimizes summaries for long-term effectiveness in streaming scenarios.
• Figure 3: Multi-interest transferability. The figure plots the performance degradation on AlignX and HiCUPID with increasing history from a secondary interest. The x-axis represents the percentage of interactions from a secondary domain interleaved into the user's primary history.
• Figure 4: RL training curves. Reward and response length of initial (left) and updated (right) preference summarizing.