Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Persona Consistency for LLMs' Role-Playing using Persona-Aware Contrastive Learning

Ke Ji, Yixin Lian, Linxu Li, Jingsheng Gao, Weiyuan Li, Bin Dai

TL;DR

The paper tackles the challenge of maintaining persona-consistent role-playing in LLMs without costly human annotations. It introduces Persona-Aware Contrastive Learning (PCL), which combines a chain-of-persona self-reflection prompt with contrastive self-play alignment to steer models toward stable, character-faithful interactions. Empirical results across open and closed models on CharacterEval show substantial improvements in role-consistency and attractiveness, with limited degradation of general knowledge and good transfer to unseen roles. The approach offers a scalable, annotation-free pathway to more coherent and engaging RPCA systems, with potential for broader deployment in personalized and entertaining AI conversations.

Abstract

In recent years, large language models (LLMs) have achieved breakthrough progress in many dialogue generation tasks. However, their lack of emotion and fine-grained role awareness limits the model's ability to provide personalized and diverse interactions further. Current methods face high costs in collecting high-quality annotated data for scenarios such as role-playing, and traditional human alignment methods are difficult to deploy due to the inherent diversity of model behavior in role-playing scenarios. Inspired by the alignment of models for safety behaviors through RLHF (Reinforcement Learning from Human Feedback), in this paper, we revisit model role-playing behavior from the perspective of persona alignment and propose a novel annotation-free framework named \textbf{\underline{P}}ersona-Aware \textbf{\underline{C}}ontrastive \textbf{\underline{L}}earning (PCL) to align LLMs' behavior during role-playing, enhancing the model's role consistency. Specifically, we first design a role chain method to encourage the model to self-question based on the role characteristics and dialogue context to adjust personality consistency. Then, we further enhance the model's role-playing strategy through iterative contrastive learning between the use of role characteristics and not. Experiments on both black-box and white-box LLMs show that LLMs equipped with PCL significantly outperform vanilla LLMs under automatic evaluation methods (CharEval \& GPT-4) and human expert evaluation.

Enhancing Persona Consistency for LLMs' Role-Playing using Persona-Aware Contrastive Learning

TL;DR

The paper tackles the challenge of maintaining persona-consistent role-playing in LLMs without costly human annotations. It introduces Persona-Aware Contrastive Learning (PCL), which combines a chain-of-persona self-reflection prompt with contrastive self-play alignment to steer models toward stable, character-faithful interactions. Empirical results across open and closed models on CharacterEval show substantial improvements in role-consistency and attractiveness, with limited degradation of general knowledge and good transfer to unseen roles. The approach offers a scalable, annotation-free pathway to more coherent and engaging RPCA systems, with potential for broader deployment in personalized and entertaining AI conversations.

Abstract

In recent years, large language models (LLMs) have achieved breakthrough progress in many dialogue generation tasks. However, their lack of emotion and fine-grained role awareness limits the model's ability to provide personalized and diverse interactions further. Current methods face high costs in collecting high-quality annotated data for scenarios such as role-playing, and traditional human alignment methods are difficult to deploy due to the inherent diversity of model behavior in role-playing scenarios. Inspired by the alignment of models for safety behaviors through RLHF (Reinforcement Learning from Human Feedback), in this paper, we revisit model role-playing behavior from the perspective of persona alignment and propose a novel annotation-free framework named \textbf{\underline{P}}ersona-Aware \textbf{\underline{C}}ontrastive \textbf{\underline{L}}earning (PCL) to align LLMs' behavior during role-playing, enhancing the model's role consistency. Specifically, we first design a role chain method to encourage the model to self-question based on the role characteristics and dialogue context to adjust personality consistency. Then, we further enhance the model's role-playing strategy through iterative contrastive learning between the use of role characteristics and not. Experiments on both black-box and white-box LLMs show that LLMs equipped with PCL significantly outperform vanilla LLMs under automatic evaluation methods (CharEval \& GPT-4) and human expert evaluation.

Paper Structure

This paper contains 35 sections, 6 equations, 4 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Related Work
  4. Human Alignment
  5. Chain-of-Thought
  6. Role-Playing
  7. Method
  8. Overview
  9. Chain of Persona Design
  10. Persona-Aware Training
  11. Contrastive Self-Play Alignment
  12. Experiments
  13. Experiments Setup
  14. Dataset and Experiments Setting
  15. General setting
  16. ...and 20 more sections

Figures (4)

  • Figure 1: Our proposed PCL adopts chain of persona, and generates contrastive samples by removing the necessary role profile for self-play alignment. This self-play process is conducted iteratively through different epochs.
  • Figure 2: The prompt for generating self-reflection persona constraints and the final response.
  • Figure 3: The criteria for expert evaluation.
  • Figure 4: The prompt of in-context learning for generating role-playing response.