CARD: Cluster-level Adaptation with Reward-guided Decoding for Personalized Text Generation

TL;DR

CARD addresses the challenge of scalable personalized text generation by combining hierarchical group priors with per-user decoding-time signals. The method partitions users into clusters and learns cluster-specific LoRA adapters $\Theta_c^{LoRA}$, while injecting a lightweight per-user vector $\lambda_u$ and a reward-guided logit correction to steer decoding without touching the backbone. Personalization is trained via a Bradley–Terry style pairwise objective on inputs paired with intra-cluster baselines, enabling robust supervision under weak signals. On LaMP and LongLaMP, CARD achieves competitive or superior generation quality with improved efficiency and cold-start robustness, making it practical for deployment at scale.

Abstract

Adapting large language models to individual users remains challenging due to the tension between fine-grained personalization and scalable deployment. We present CARD, a hierarchical framework that achieves effective personalization through progressive refinement. CARD first clusters users according to shared stylistic patterns and learns cluster-specific LoRA adapters, enabling robust generalization and strong low-resource performance. To capture individual differences within each cluster, we propose an implicit preference learning mechanism that contrasts user-authored text with cluster-level generations, allowing the model to infer user-specific style preferences without manual annotation. At inference time, CARD injects personalization exclusively at decoding via lightweight user preference vectors and low-rank logit corrections, while keeping the base model frozen. Experiments on the LaMP and LongLaMP benchmarks show that CARD achieves competitive or superior generation quality compared to state-of-the-art baselines, while significantly improving efficiency and scalability for practical personalized text generation.

Figures (6)

• Figure 1: Overview of the CARD framework. * Purple components correspond to group-level personalization and orange components represent user-level personalization. Solid black arrows indicate the execution flow of the model, while blue text denotes the flow of data.
• Figure 2: A case study for LaMP-7. The gray background highlights the longest contiguous span shared with the reference/user's answer, while the yellow background highlights content overlapping with the user history.
• Figure 3: CARD performance comparison for low resource setting.
• Figure 4: User vector personalization strength.
• Figure 5: Performance via different $K$ group clusters.