Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CURP: Codebook-based Continuous User Representation for Personalized Generation with LLMs

Liang Wang, Xinyi Mou, Xiaoyou Liu, Xuanjing Huang, Zhongyu Wei

TL;DR

CURP introduces a codebook-based continuous user representation for personalized generation with LLMs. By encoding user histories with a bidirectional encoder and compressing into a discrete prototype codebook via Product Quantization, CURP achieves noise-robust, data-efficient personalization with a compact footprint of about 20M trainable parameters. The two-stage training—Prototype Codebook Construction and Prototype Behavior Aligning—yields a decoder-agnostic framework that generalizes across tasks and supports cloud-edge deployment using non-invertible discrete indices. Across Reddit QA, News Headline, Tweet Paraphrase, and Review Writing, CURP outperforms strong baselines on diverse metrics and demonstrates interpretable prototypes and robust cross-task generalization. The work advances practical, scalable personalization for generation with LLMs, balancing fidelity, efficiency, and privacy.

Abstract

User modeling characterizes individuals through their preferences and behavioral patterns to enable personalized simulation and generation with Large Language Models (LLMs) in contemporary approaches. However, existing methods, whether prompt-based or training-based methods, face challenges in balancing personalization quality against computational and data efficiency. We propose a novel framework CURP, which employs a bidirectional user encoder and a discrete prototype codebook to extract multi-dimensional user traits. This design enables plug-and-play personalization with a small number of trainable parameters (about 20M parameters, about 0.2\% of the total model size). Through extensive experiments on variant generation tasks, we show that CURP achieves superior performance and generalization compared to strong baselines, while offering better interpretability and scalability. The code are available at https://github.com/RaidonWong/CURP_code

CURP: Codebook-based Continuous User Representation for Personalized Generation with LLMs

TL;DR

CURP introduces a codebook-based continuous user representation for personalized generation with LLMs. By encoding user histories with a bidirectional encoder and compressing into a discrete prototype codebook via Product Quantization, CURP achieves noise-robust, data-efficient personalization with a compact footprint of about 20M trainable parameters. The two-stage training—Prototype Codebook Construction and Prototype Behavior Aligning—yields a decoder-agnostic framework that generalizes across tasks and supports cloud-edge deployment using non-invertible discrete indices. Across Reddit QA, News Headline, Tweet Paraphrase, and Review Writing, CURP outperforms strong baselines on diverse metrics and demonstrates interpretable prototypes and robust cross-task generalization. The work advances practical, scalable personalization for generation with LLMs, balancing fidelity, efficiency, and privacy.

Abstract

User modeling characterizes individuals through their preferences and behavioral patterns to enable personalized simulation and generation with Large Language Models (LLMs) in contemporary approaches. However, existing methods, whether prompt-based or training-based methods, face challenges in balancing personalization quality against computational and data efficiency. We propose a novel framework CURP, which employs a bidirectional user encoder and a discrete prototype codebook to extract multi-dimensional user traits. This design enables plug-and-play personalization with a small number of trainable parameters (about 20M parameters, about 0.2\% of the total model size). Through extensive experiments on variant generation tasks, we show that CURP achieves superior performance and generalization compared to strong baselines, while offering better interpretability and scalability. The code are available at https://github.com/RaidonWong/CURP_code
Paper Structure (38 sections, 11 equations, 4 figures, 6 tables)

This paper contains 38 sections, 11 equations, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. User Modeling
  4. LLM-based Personalization
  5. Methodology
  6. Model Components
  7. User Encoder
  8. Codebook
  9. Decoder
  10. Training Procedure
  11. Stage 1: Prototype Codebook Construction
  12. Stage 2: Prototype Behavior Aligning
  13. Training details
  14. Experimental Setup
  15. Comparison Methods
  16. ...and 23 more sections

Figures (4)

  • Figure 1: Different paradigm for user response simulation. (a) non-parametric prompting: prompting LLMs with text-based user behaviors, generalizable but noisy; (b) parametric training: Finetuning a PEFT module or an embedding for each user, accurate but expensive; (c) Ours: adopting prototype combination based user embedding via a codebook to guide user content generation, achieve a balance between two paradigms.
  • Figure 2: The overall framework of our proposed CURP. Our training procedure can be divided into two stages: Prototype Codebook Construction (PCC) and Prototype Behaviour Aligning (PBA).
  • Figure 3: (a): PQ quantization significantly reduces history similarity compared to raw embeddings, aiding prototype disentanglement. (b): Different initialization strategies affect codebook usage and combination diversity. (c)(d): Codebook size and PQ subspace comparisons reveal that larger vocabularies or more subspaces do not yield significant performance gains, with optimal balance achieved at moderate configurations.
  • Figure 4: Analysis: The impact of history numbers.