Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Guided Profile Generation Improves Personalization with LLMs

Jiarui Zhang

TL;DR

Guided Profile Generation is proposed, a general method designed to generate personal profiles in natural language that improves LLM's personalization ability across different tasks, and increases 37% accuracy in predicting personal preference compared to directly feeding the LLMs with raw personal context.

Abstract

In modern commercial systems, including Recommendation, Ranking, and E-Commerce platforms, there is a trend towards improving customer experiences by incorporating Personalization context as input into Large Language Models (LLMs). However, LLMs often struggle to effectively parse and utilize sparse and complex personal context without additional processing or contextual enrichment, underscoring the need for more sophisticated context understanding mechanisms. In this work, we propose Guided Profile Generation (GPG), a general method designed to generate personal profiles in natural language. As is observed, intermediate guided profile generation enables LLMs to summarize, and extract the important, distinctive features from the personal context into concise, descriptive sentences, precisely tailoring their generation more closely to an individual's unique habits and preferences. Our experimental results show that GPG improves LLM's personalization ability across different tasks, for example, it increases 37% accuracy in predicting personal preference compared to directly feeding the LLMs with raw personal context.

Guided Profile Generation Improves Personalization with LLMs

TL;DR

Guided Profile Generation is proposed, a general method designed to generate personal profiles in natural language that improves LLM's personalization ability across different tasks, and increases 37% accuracy in predicting personal preference compared to directly feeding the LLMs with raw personal context.

Abstract

In modern commercial systems, including Recommendation, Ranking, and E-Commerce platforms, there is a trend towards improving customer experiences by incorporating Personalization context as input into Large Language Models (LLMs). However, LLMs often struggle to effectively parse and utilize sparse and complex personal context without additional processing or contextual enrichment, underscoring the need for more sophisticated context understanding mechanisms. In this work, we propose Guided Profile Generation (GPG), a general method designed to generate personal profiles in natural language. As is observed, intermediate guided profile generation enables LLMs to summarize, and extract the important, distinctive features from the personal context into concise, descriptive sentences, precisely tailoring their generation more closely to an individual's unique habits and preferences. Our experimental results show that GPG improves LLM's personalization ability across different tasks, for example, it increases 37% accuracy in predicting personal preference compared to directly feeding the LLMs with raw personal context.
Paper Structure (20 sections, 4 figures, 6 tables)

This paper contains 20 sections, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Guided Profile Generation
  4. High-level Workflow
  5. Evaluation Tasks and Metrics
  6. Task of Preference Prediction
  7. Task of Text Paraphrasing
  8. Task of Dialogue Response Generation
  9. Experiments
  10. Baselines.
  11. GPG Specifications.
  12. Experimental Results
  13. Analysis and Discussions
  14. Ablation Studies
  15. Error analysis and Observations.
  16. ...and 5 more sections

Figures (4)

  • Figure 1: A motivating example. The model is given a personal context reflecting the person's writing style, and the task is to paraphrase a new tweet for the user. We show gpt-3.5-turbo-1106's response under different input conditions. The result shows that generating a descriptive personal profile with proper guidance helps the model finish the personalization better.
  • Figure 2: Illustration of GPG described in Section \ref{['sec:gpg']}: Given a personal context, we instruct LLM to generate a descriptive personal profile via self-guidance. The personal profile is then used to complete the personal task. GPG enables LLM to generate high-quality personal profiles, improving their performance on personalization. Note that our experiments are conducted in textual domain, images are for illustrative purposes.
  • Figure 3: Text paraphrasing on Twitter performance of GPG in comparison with direct generation without personal context (DG w/o PC), direct generation with personal context (DG w/ PC) and Profile Generation (PG).
  • Figure 4: Examples of personal context, personal context digestion, and personal profile of three tasks under our test. We select only part of the personal context due to their length.