Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Character-R1: Enhancing Role-Aware Reasoning in Role-Playing Agents via RLVR

Yihong Tang, Kehai Chen, Xuefeng Bai, Benyou Wang, Zeming Liu, Haifeng Wang, Min Zhang

TL;DR

Character-R1 introduces a verifiable, cognitivist reinforcement learning framework for role-playing agents, combining Cognitive Focus, Reference Guidance, and Character-Conditional Reward Normalization to enforce robust internal reasoning aligned with character personas. The method builds on GRPO, enabling stable updates without a value function, and uses explicit focus signals and overlap-based guidance to shape thinking and output. Empirical results on CharacterBench and SocialBench show state-of-the-art performance in knowledge, memory, and persona consistency, with strong cross-lingual generalization and favorable ablations. The work demonstrates that explicit, multi-dimensional rewards can significantly improve character fidelity and immersive interactions in RPAs, offering practical benefits for entertainment, social simulation, and digital companionship applications.

Abstract

Current role-playing agents (RPAs) are typically constructed by imitating surface-level behaviors, but this approach lacks internal cognitive consistency, often causing out-of-character errors in complex situations. To address this, we propose Character-R1, a framework designed to provide comprehensive verifiable reward signals for effective role-aware reasoning, which are missing in recent studies. Specifically, our framework comprises three core designs: (1) Cognitive Focus Reward, which enforces explicit label-based analysis of 10 character elements (e.g., worldview) to structure internal cognition; (2) Reference-Guided Reward, which utilizes overlap-based metrics with reference responses as optimization anchors to enhance exploration and performance; and (3) Character-Conditioned Reward Normalization, which adjusts reward distributions based on character categories to ensure robust optimization across heterogeneous roles. Extensive experiments demonstrate that Character-R1 significantly outperforms existing methods in knowledge, memory and others.

Character-R1: Enhancing Role-Aware Reasoning in Role-Playing Agents via RLVR

TL;DR

Character-R1 introduces a verifiable, cognitivist reinforcement learning framework for role-playing agents, combining Cognitive Focus, Reference Guidance, and Character-Conditional Reward Normalization to enforce robust internal reasoning aligned with character personas. The method builds on GRPO, enabling stable updates without a value function, and uses explicit focus signals and overlap-based guidance to shape thinking and output. Empirical results on CharacterBench and SocialBench show state-of-the-art performance in knowledge, memory, and persona consistency, with strong cross-lingual generalization and favorable ablations. The work demonstrates that explicit, multi-dimensional rewards can significantly improve character fidelity and immersive interactions in RPAs, offering practical benefits for entertainment, social simulation, and digital companionship applications.

Abstract

Current role-playing agents (RPAs) are typically constructed by imitating surface-level behaviors, but this approach lacks internal cognitive consistency, often causing out-of-character errors in complex situations. To address this, we propose Character-R1, a framework designed to provide comprehensive verifiable reward signals for effective role-aware reasoning, which are missing in recent studies. Specifically, our framework comprises three core designs: (1) Cognitive Focus Reward, which enforces explicit label-based analysis of 10 character elements (e.g., worldview) to structure internal cognition; (2) Reference-Guided Reward, which utilizes overlap-based metrics with reference responses as optimization anchors to enhance exploration and performance; and (3) Character-Conditioned Reward Normalization, which adjusts reward distributions based on character categories to ensure robust optimization across heterogeneous roles. Extensive experiments demonstrate that Character-R1 significantly outperforms existing methods in knowledge, memory and others.
Paper Structure (32 sections, 2 equations, 8 figures, 7 tables)

This paper contains 32 sections, 2 equations, 8 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Role-Playing Agents with LLMs
  4. Large Reasoning Models
  5. Method
  6. Preliminaries
  7. Cognitive Focus
  8. Reference Guidance
  9. Character-Conditional Reward Normalization
  10. Experiments
  11. Experimental Setup
  12. Benchmark and Datasets
  13. Baselines
  14. Implementation
  15. Main Results
  16. ...and 17 more sections

Figures (8)

  • Figure 1: Comparison of role-playing paradigms. (a) Behaviorism mimics surface patterns,leading to OOC behavior; (b) Early cognitivism uses indirect,partial,and fragile rewards;(c) Character-R1 employs direct,comprehensive,and flexible rewards to explicitly shape the character's internal cognitive state.
  • Figure 2: Overview of the Character-R1 methodology. The layout illustrates the generation of the reasoning trajectory ($y_{CoT}$) containing explicit verifiable <focus> tags, followed by the response ($y_{ans}$). Rewards are derived from three sources: Cognitive Focus accuracy, Focus Attribute quality, and Reference Guidance. These rewards undergo Character-Conditional Normalization utilizing Role Grouping statistics before being aggregated to update the policy via GRPO.
  • Figure 3: The t-SNE visualization of hidden states of the model under different cognitive focus task.
  • Figure 4: Human evaluation.
  • Figure 5: Training reward curves.
  • ...and 3 more figures