Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rethinking Personalization in Large Language Models at the Token Level

Chenheng Zhang, Yijun Lu, Lizhe Fang, Chunyuan Zheng, Jiajun Chai, Xiaohan Wang, Guojun Yin, Wei Lin, Yisen Wang, Zhouchen Lin

TL;DR

PerContrast, a self-contrast method that estimates each output token's dependence on user-specific information through causal intervention, is proposed and developed, which adaptively upweights tokens with higher estimated personalization degrees during training via a bootstrap procedure, enabling the model to alternate between estimating and optimizing these tokens.

Abstract

With large language models (LLMs) now performing strongly across diverse tasks, there is growing demand for them to personalize outputs for individual users. Personalization is typically framed as an additional layer on top of a base NLP task, requiring model responses to meet user-specific needs while still accomplishing the underlying task. From a token-level perspective, different tokens in a response contribute to personalization to varying degrees. Tokens with higher personalization relevance should therefore receive greater emphasis when developing personalized LLMs. However, accurately estimating such personalization degrees remains challenging. To address this challenge, we propose PerContrast, a self-contrast method that estimates each output token's dependence on user-specific information through causal intervention. Building on this mechanism, we develop the PerCE loss, which adaptively upweights tokens with higher estimated personalization degrees during training via a bootstrap procedure, enabling the model to alternate between estimating and optimizing these tokens. Experiments on multiple LLMs demonstrate that PerCE substantially improves personalization performance with minimal additional cost, achieving average gains of over 10% and up to 68.04% on the LongLaMP dataset, along with strong cross-task and cross-scenario transferability. These results highlight the importance of token-level personalization modeling and establish token-aware training as a simple yet effective paradigm for advancing personalized LLMs.

Rethinking Personalization in Large Language Models at the Token Level

TL;DR

PerContrast, a self-contrast method that estimates each output token's dependence on user-specific information through causal intervention, is proposed and developed, which adaptively upweights tokens with higher estimated personalization degrees during training via a bootstrap procedure, enabling the model to alternate between estimating and optimizing these tokens.

Abstract

With large language models (LLMs) now performing strongly across diverse tasks, there is growing demand for them to personalize outputs for individual users. Personalization is typically framed as an additional layer on top of a base NLP task, requiring model responses to meet user-specific needs while still accomplishing the underlying task. From a token-level perspective, different tokens in a response contribute to personalization to varying degrees. Tokens with higher personalization relevance should therefore receive greater emphasis when developing personalized LLMs. However, accurately estimating such personalization degrees remains challenging. To address this challenge, we propose PerContrast, a self-contrast method that estimates each output token's dependence on user-specific information through causal intervention. Building on this mechanism, we develop the PerCE loss, which adaptively upweights tokens with higher estimated personalization degrees during training via a bootstrap procedure, enabling the model to alternate between estimating and optimizing these tokens. Experiments on multiple LLMs demonstrate that PerCE substantially improves personalization performance with minimal additional cost, achieving average gains of over 10% and up to 68.04% on the LongLaMP dataset, along with strong cross-task and cross-scenario transferability. These results highlight the importance of token-level personalization modeling and establish token-aware training as a simple yet effective paradigm for advancing personalized LLMs.
Paper Structure (43 sections, 1 theorem, 11 equations, 7 figures, 17 tables)

This paper contains 43 sections, 1 theorem, 11 equations, 7 figures, 17 tables.

Table of Contents

  1. Introduction
  2. PerContrast: Causal Measurement of Token-Level Personalization
  3. Measuring Tokens via Self-Contrast
  4. Personalization Intervention.
  5. A Causal-Theoretic Analysis of PIR
  6. PerCE: Enhancing Personalized LLMs with an EM Perspective
  7. Weighted CE as a More Suitable Objective.
  8. PIR as an Empirical Estimator of Token Importance.
  9. Causal Interpretation for token reweighting.
  10. An EM Interpretation of PerCE.
  11. Experiments
  12. Baseline.
  13. Dataset.
  14. Metric.
  15. Setup.
  16. ...and 28 more sections

Key Result

Theorem 2.3

Under Assumptions assump:sutva-assump:condindep, the token-level causal effect is the same as the proposed PIR

Figures (7)

  • Figure 1: Tokens contribute to personalization to varying degrees, and their contribution distributions differ across tasks. In writing tasks, stylistic tokens play a more prominent role, whereas in conversational tasks, information-bearing tokens are more important.
  • Figure 2: Illustration of PerContrast. By intervening on the user persona, PerContrast estimates the personalization degree of each output token. PIR denotes the Personal Influence Ratio (defined in Equation \ref{['PIR']}.
  • Figure 3: Causal Graph When Predict $Y_i$
  • Figure 4: Performance comparison across three methods on personal tokens identification.
  • Figure 5: Distribution of PIR scores across tokens. Tokens with a PIR score above 1 are classified as personal tokens. The figure shows the proportions of correct and incorrect classifications within different PIR score ranges.
  • ...and 2 more figures

Theorems & Definitions (1)

  • Theorem 2.3: Relation Between Causal Effect and PIR