Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PersonaDual: Balancing Personalization and Objectivity via Adaptive Reasoning

Xiaoyou Liu, Xinyi Mou, Shengbin Yue, Liang Wang, Yuqing Wang, Qiexiang Wang, Tianrui Qin, Wangchunshu Zhou, Zhongyu Wei

TL;DR

PersonaDual tackles the challenge of balancing personalization with objectivity in LLMs by enabling adaptive switching between two reasoning modes: general objective reasoning and personalized reasoning. It introduces a two-stage training framework: supervised fine-tuning to instill both modes, followed by DualGRPO, a prefix-aware reinforcement learning algorithm that optimizes context-sensitive mode selection. Empirical results show PersonaDual nearly neutralizes the negative impact of unaligned personalization on objective QA and even yields gains when personas are aligned, while preserving personalized benefits on subjective tasks. This approach advances reliable, user-adaptive AI by reducing personalization-induced errors and enabling beneficial personalization in a principled, controllable way.

Abstract

As users increasingly expect LLMs to align with their preferences, personalized information becomes valuable. However, personalized information can be a double-edged sword: it can improve interaction but may compromise objectivity and factual correctness, especially when it is misaligned with the question. To alleviate this problem, we propose PersonaDual, a framework that supports both general-purpose objective reasoning and personalized reasoning in a single model, and adaptively switches modes based on context. PersonaDual is first trained with SFT to learn two reasoning patterns, and then further optimized via reinforcement learning with our proposed DualGRPO to improve mode selection. Experiments on objective and personalized benchmarks show that PersonaDual preserves the benefits of personalization while reducing interference, achieving near interference-free performance and better leveraging helpful personalized signals to improve objective problem-solving.

PersonaDual: Balancing Personalization and Objectivity via Adaptive Reasoning

TL;DR

PersonaDual tackles the challenge of balancing personalization with objectivity in LLMs by enabling adaptive switching between two reasoning modes: general objective reasoning and personalized reasoning. It introduces a two-stage training framework: supervised fine-tuning to instill both modes, followed by DualGRPO, a prefix-aware reinforcement learning algorithm that optimizes context-sensitive mode selection. Empirical results show PersonaDual nearly neutralizes the negative impact of unaligned personalization on objective QA and even yields gains when personas are aligned, while preserving personalized benefits on subjective tasks. This approach advances reliable, user-adaptive AI by reducing personalization-induced errors and enabling beneficial personalization in a principled, controllable way.

Abstract

As users increasingly expect LLMs to align with their preferences, personalized information becomes valuable. However, personalized information can be a double-edged sword: it can improve interaction but may compromise objectivity and factual correctness, especially when it is misaligned with the question. To alleviate this problem, we propose PersonaDual, a framework that supports both general-purpose objective reasoning and personalized reasoning in a single model, and adaptively switches modes based on context. PersonaDual is first trained with SFT to learn two reasoning patterns, and then further optimized via reinforcement learning with our proposed DualGRPO to improve mode selection. Experiments on objective and personalized benchmarks show that PersonaDual preserves the benefits of personalization while reducing interference, achieving near interference-free performance and better leveraging helpful personalized signals to improve objective problem-solving.
Paper Structure (39 sections, 5 equations, 5 figures, 8 tables)

This paper contains 39 sections, 5 equations, 5 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Impact of Personalization on Factuality
  4. Adaptive Inference and Dual-Process Architectures
  5. Methods
  6. Framework Overview
  7. Stage 1: Mixed Mode Integration
  8. General Purpose Objective Reasoning Mode.
  9. Personalized Reasoning Mode.
  10. Stage 2: Adaptive Mode Selection
  11. Prefix Forced Sampling.
  12. Dual-Mode Advantage Decomposition.
  13. Experimental Setup
  14. Training Details.
  15. Baselines.
  16. ...and 24 more sections

Figures (5)

  • Figure 1: Effects of personalized information under different persona settings. We report the averaged results of DeepSeek-R1 guo2025deepseek and Qwen3-30B-A3B-Thinking yang2025qwen3 for general models, and ALIGNXPERT-ICA li20251 and ALIGNXPERT-PBA li20251 for personalized models. Positive (negative) values indicate performance gains (drops) compared to the no-persona setting (indicated by the zero line).
  • Figure 2: Overview of PersonaDual Framework. PersonaDual equips a single LLM with two complementary response modes, objective and personalized reasoning, and learns to adaptively select between them based on the user query and available persona. Training follows a two-stage paradigm: supervised learning to disentangle the two reasoning behaviors, followed by DualGRPO to refine context-aware mode selection. This design captures the benefits of aligned persona cues while mitigating interference from irrelevant or unaligned personas, improving both objective correctness and personalization quality.
  • Figure 3: (a) Performance comparison under mixed task settings with varying personalization ratios. (b) Comparison of mode proportions in dual-mode models. PD is short for PersonaDual.
  • Figure 4: Attention heatmap visualizing which persona keywords influence mode selection in PersonaDual. Darker colors indicate higher attention weights.
  • Figure 5: Extracted keywords from personas