Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

"In Dialogues We Learn": Towards Personalized Dialogue Without Pre-defined Profiles through In-Dialogue Learning

Chuanqi Cheng, Quan Tu, Shuo Shang, Cunli Mao, Zhengtao Yu, Wei Wu, Rui Yan

TL;DR

This work tackles personalized dialogue generation without pre-defined user profiles by learning persona information directly from dialogue history through In-Dialogue Learning (IDL). IDL operates in two stages—Mutual Supervised Learning (MSL) to reorganize and leverage dialogue sessions via Static and Dynamic Persona Identification, and Deep Personalized Alignment (DPA) with Direct Preference Optimization with Criterion (DPOC) to closely align responses with the learned persona. It introduces novel components including convED-based reordering, a persona extractor-based clustering, and criterion-driven data construction to mitigate preference degradation, achieving substantial improvements on multiple benchmarks (up to $200\%$ BLEU and $247\%$ ROUGE gains) and strong human validation. The method reduces reliance on labor-intensive profiles and demonstrates effective persona learning from multi-turn dialogues, offering a scalable approach for personalized dialogue in large language models with practical impact for adaptive conversation systems.

Abstract

Personalized dialogue systems have gained significant attention in recent years for their ability to generate responses in alignment with different personas. However, most existing approaches rely on pre-defined personal profiles, which are not only time-consuming and labor-intensive to create but also lack flexibility. We propose In-Dialogue Learning (IDL), a fine-tuning framework that enhances the ability of pre-trained large language models to leverage dialogue history to characterize persona for completing personalized dialogue generation tasks without pre-defined profiles. Our experiments on three datasets demonstrate that IDL brings substantial improvements, with BLEU and ROUGE scores increasing by up to 200% and 247%, respectively. Additionally, the results of human evaluations further validate the efficacy of our proposed method.

"In Dialogues We Learn": Towards Personalized Dialogue Without Pre-defined Profiles through In-Dialogue Learning

TL;DR

This work tackles personalized dialogue generation without pre-defined user profiles by learning persona information directly from dialogue history through In-Dialogue Learning (IDL). IDL operates in two stages—Mutual Supervised Learning (MSL) to reorganize and leverage dialogue sessions via Static and Dynamic Persona Identification, and Deep Personalized Alignment (DPA) with Direct Preference Optimization with Criterion (DPOC) to closely align responses with the learned persona. It introduces novel components including convED-based reordering, a persona extractor-based clustering, and criterion-driven data construction to mitigate preference degradation, achieving substantial improvements on multiple benchmarks (up to BLEU and ROUGE gains) and strong human validation. The method reduces reliance on labor-intensive profiles and demonstrates effective persona learning from multi-turn dialogues, offering a scalable approach for personalized dialogue in large language models with practical impact for adaptive conversation systems.

Abstract

Personalized dialogue systems have gained significant attention in recent years for their ability to generate responses in alignment with different personas. However, most existing approaches rely on pre-defined personal profiles, which are not only time-consuming and labor-intensive to create but also lack flexibility. We propose In-Dialogue Learning (IDL), a fine-tuning framework that enhances the ability of pre-trained large language models to leverage dialogue history to characterize persona for completing personalized dialogue generation tasks without pre-defined profiles. Our experiments on three datasets demonstrate that IDL brings substantial improvements, with BLEU and ROUGE scores increasing by up to 200% and 247%, respectively. Additionally, the results of human evaluations further validate the efficacy of our proposed method.
Paper Structure (33 sections, 12 equations, 5 figures, 7 tables)

This paper contains 33 sections, 12 equations, 5 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Personalized Dialogue Systems
  4. In-Context Learning
  5. Method
  6. Problem Formalization
  7. Mutual Supervised Learning
  8. Static Persona Identification
  9. Dynamic Persona Identification
  10. Deep Personalized Alignment
  11. DPOC
  12. Data Construction
  13. Experiments
  14. Datasets
  15. Baselines
  16. ...and 18 more sections

Figures (5)

  • Figure 1: An example of profile-free personalized dialogue generation by In-Dialogue Learning. Persona information in different dialogues is marked with corresponding colors.
  • Figure 2: The framework of IDL. Left: the MSL stage that fine-tunes the dialogue model using data organized by static persona and dynamic persona identification. Right: the DPA stage in which we collect three types of criterion examples and conduct DPOC to further optimize the model to align with the target persona in a better way.
  • Figure 3: Human evaluation results for IDL compared to ICL. Both methods adopt LLaMA-2-13B-Chat.
  • Figure 4: Experiments with different numbers of dialogue sessions on the Movie and LIGHT.
  • Figure 5: A case study. Keywords in the profile are marked in red, while the corresponding keywords that have high attention weight within dialogue sessions are bolded and highlighted with a yellow background.