Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OpenCharacter: Training Customizable Role-Playing LLMs with Large-Scale Synthetic Personas

Xiaoyang Wang, Hongming Zhang, Tao Ge, Wenhao Yu, Dian Yu, Dong Yu

TL;DR

This work introduces OpenCharacter, a data-synthesis framework to train customizable LLMs for out-of-domain character generalization in role-playing dialogue. It builds a large synthetic character library from Persona Hub, enriches profiles with fine-grained grounding, and leverages two strategies—response rewriting (OpenCharacter-R) and response generation (OpenCharacter-G)—to produce high-quality instruction-following data used for SFT on LLaMA-3-8B. The approach yields strong performance on PersonaGym benchmarks, surpassing several GPT-3.5/4o baselines and approaching GPT-4o, with public release of 20k synthetic characters and ~319k dialogue samples. The study demonstrates the viability of scalable synthetic personas for character generalization and discusses future directions including larger backbones and grounding knowledge in virtual-world contexts.

Abstract

Customizable role-playing in large language models (LLMs), also known as character generalization, is gaining increasing attention for its versatility and cost-efficiency in developing and deploying role-playing dialogue agents. This study explores a large-scale data synthesis approach to equip LLMs with character generalization capabilities. We begin by synthesizing large-scale character profiles using personas from Persona Hub and then explore two strategies: response rewriting and response generation, to create character-aligned instructional responses. To validate the effectiveness of our synthetic instruction tuning data for character generalization, we perform supervised fine-tuning (SFT) using the LLaMA-3 8B model. Our best-performing model strengthens the original LLaMA-3 8B Instruct model and achieves performance comparable to GPT-4o models on role-playing dialogue. We release our synthetic characters and instruction-tuning dialogues to support public research.

OpenCharacter: Training Customizable Role-Playing LLMs with Large-Scale Synthetic Personas

TL;DR

This work introduces OpenCharacter, a data-synthesis framework to train customizable LLMs for out-of-domain character generalization in role-playing dialogue. It builds a large synthetic character library from Persona Hub, enriches profiles with fine-grained grounding, and leverages two strategies—response rewriting (OpenCharacter-R) and response generation (OpenCharacter-G)—to produce high-quality instruction-following data used for SFT on LLaMA-3-8B. The approach yields strong performance on PersonaGym benchmarks, surpassing several GPT-3.5/4o baselines and approaching GPT-4o, with public release of 20k synthetic characters and ~319k dialogue samples. The study demonstrates the viability of scalable synthetic personas for character generalization and discusses future directions including larger backbones and grounding knowledge in virtual-world contexts.

Abstract

Customizable role-playing in large language models (LLMs), also known as character generalization, is gaining increasing attention for its versatility and cost-efficiency in developing and deploying role-playing dialogue agents. This study explores a large-scale data synthesis approach to equip LLMs with character generalization capabilities. We begin by synthesizing large-scale character profiles using personas from Persona Hub and then explore two strategies: response rewriting and response generation, to create character-aligned instructional responses. To validate the effectiveness of our synthetic instruction tuning data for character generalization, we perform supervised fine-tuning (SFT) using the LLaMA-3 8B model. Our best-performing model strengthens the original LLaMA-3 8B Instruct model and achieves performance comparable to GPT-4o models on role-playing dialogue. We release our synthetic characters and instruction-tuning dialogues to support public research.
Paper Structure (27 sections, 3 equations, 4 figures, 5 tables)

This paper contains 27 sections, 3 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Approach
  4. Problem Definition
  5. Character Generalization with Data Synthesis
  6. Character Profile Synthesis
  7. Character-driven Response Rewriting (OpenCharacter-R)
  8. Character-driven Response Generation (OpenCharacter-G)
  9. Supervised Fine-tuning
  10. Evaluation
  11. Evaluation with PersonaGym
  12. Evaluation with PersonaGym-Light
  13. Evaluation Model and Overall Score
  14. Experiments
  15. Data Synthesis
  16. ...and 12 more sections

Figures (4)

  • Figure 1: Our overall data synthesis approach. As an example, we start with character profile synthesis using a persona from Persona Hub, and then explore character-driven response rewriting and generation.
  • Figure 2: An example of our response synthesis. For a user's question from LIMA, we show the abbreviated version of responses that are rewritten and generated respectively through our approach illustrated in Figure \ref{['fig:approach']}. Both responses align with the given character, though the rewritten response largely keeps the knowledge details of the original response while the generated response does not typically contain the exact details.
  • Figure 3: Our data synthesis prompt for character profile synthesis to generate the enriched character profile with a persona from Persona Hub as the input.
  • Figure 5: Our model's system prompts to incorporate the user-specified persona and character profile. We further remove the character profile-related content for test scenarios without character profiles (e.g., PersonaGym).