Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

User Profile with Large Language Models: Construction, Updating, and Benchmarking

Nusrat Jahan Prottasha, Md Kowsher, Hafijur Raman, Israt Jahan Anny, Prakash Bhat, Ivan Garibay, Ozlem Garibay

TL;DR

This paper presents two high-quality open-source user profile datasets: one for profile construction and another for profile updating, and shows a methodology that uses large language models (LLMs) to tackle both profile construction and updating.

Abstract

User profile modeling plays a key role in personalized systems, as it requires building accurate profiles and updating them with new information. In this paper, we present two high-quality open-source user profile datasets: one for profile construction and another for profile updating. These datasets offer a strong basis for evaluating user profile modeling techniques in dynamic settings. We also show a methodology that uses large language models (LLMs) to tackle both profile construction and updating. Our method uses a probabilistic framework to predict user profiles from input text, allowing for precise and context-aware profile generation. Our experiments demonstrate that models like Mistral-7b and Llama2-7b perform strongly in both tasks. LLMs improve the precision and recall of the generated profiles, and high evaluation scores confirm the effectiveness of our approach.

User Profile with Large Language Models: Construction, Updating, and Benchmarking

TL;DR

This paper presents two high-quality open-source user profile datasets: one for profile construction and another for profile updating, and shows a methodology that uses large language models (LLMs) to tackle both profile construction and updating.

Abstract

User profile modeling plays a key role in personalized systems, as it requires building accurate profiles and updating them with new information. In this paper, we present two high-quality open-source user profile datasets: one for profile construction and another for profile updating. These datasets offer a strong basis for evaluating user profile modeling techniques in dynamic settings. We also show a methodology that uses large language models (LLMs) to tackle both profile construction and updating. Our method uses a probabilistic framework to predict user profiles from input text, allowing for precise and context-aware profile generation. Our experiments demonstrate that models like Mistral-7b and Llama2-7b perform strongly in both tasks. LLMs improve the precision and recall of the generated profiles, and high evaluation scores confirm the effectiveness of our approach.

Paper Structure

This paper contains 24 sections, 7 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Dataset Construction
  4. Construction User Profile Dataset
  5. Updating User Profile Dataset
  6. Problem Definition
  7. Profile Construction:
  8. Profile Updating:
  9. Model Construction
  10. Profile Construction
  11. Profile Updating
  12. Experiments
  13. Experimental Setup
  14. Large Language Models (LLMs)
  15. Evaluation Metrics
  16. ...and 9 more sections

Figures (4)

  • Figure 1: Overview of User Profile Management. Panel (a) shows profile construction from initial user data using an LLM, while panel (b) illustrates profile updating with new user information, maintaining dynamic User Profile Memory.
  • Figure 2: Top 10 Most Frequent Words. This bar chart illustrates the ten most frequently occurring words in the dataset, highlighting key terms such as "university," "born," and "first." The word "university" appears most frequently (4,545 times), followed by "born" (3,932) and "first" (3,614). The distribution suggests a strong focus on biographical and educational information within the user profiles.
  • Figure 3: Distribution of Various Attributes in User Information
  • Figure 4: Visualization of the dataset: (a) Word cloud before constructing user profile dataset; (b) Word cloud after constructing user profile dataset.