Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PCoKG: Personality-aware Commonsense Reasoning with Debate

Weijie Li, Zhongqing Wang, Guodong Zhou

TL;DR

PCoKG tackles the lack of personality-aware reasoning in commonsense systems by introducing a large-scale knowledge graph that encodes (event, MBTI, reasoning, outcome) quadruples. It builds the dataset through a two-stage LLM-driven pipeline: evaluator-filtered event–dimension extraction from ATOMIC and a role-based debate among Proponent/Opponent/Judge to produce MBTI-aligned, high-quality inferences across $9$ dimensions for all $16$ MBTI types, totaling $521{,}316$ quadruples. The authors validate the dataset with multi-faceted analyses, ablations, and cross-model fine-tuning, showing robustness and improved performance as model scale increases. They demonstrate practical value in persona-based dialogue, where PCoKG yields more consistent and context-aware responses, highlighting its potential for personalized AI systems.

Abstract

Most commonsense reasoning models overlook the influence of personality traits, limiting their effectiveness in personalized systems such as dialogue generation. To address this limitation, we introduce the Personality-aware Commonsense Knowledge Graph (PCoKG), a structured dataset comprising 521,316 quadruples. We begin by employing three evaluators to score and filter events from the ATOMIC dataset, selecting those that are likely to elicit diverse reasoning patterns across different personality types. For knowledge graph construction, we leverage the role-playing capabilities of large language models (LLMs) to perform reasoning tasks. To enhance the quality of the generated knowledge, we incorporate a debate mechanism consisting of a proponent, an opponent, and a judge, which iteratively refines the outputs through feedback loops. We evaluate the dataset from multiple perspectives and conduct fine-tuning and ablation experiments using multiple LLM backbones to assess PCoKG's robustness and the effectiveness of its construction pipeline. Our LoRA-based fine-tuning results indicate a positive correlation between model performance and the parameter scale of the base models. Finally, we apply PCoKG to persona-based dialogue generation, where it demonstrates improved consistency between generated responses and reference outputs. This work bridges the gap between commonsense reasoning and individual cognitive differences, enabling the development of more personalized and context-aware AI systems.

PCoKG: Personality-aware Commonsense Reasoning with Debate

TL;DR

PCoKG tackles the lack of personality-aware reasoning in commonsense systems by introducing a large-scale knowledge graph that encodes (event, MBTI, reasoning, outcome) quadruples. It builds the dataset through a two-stage LLM-driven pipeline: evaluator-filtered event–dimension extraction from ATOMIC and a role-based debate among Proponent/Opponent/Judge to produce MBTI-aligned, high-quality inferences across dimensions for all MBTI types, totaling quadruples. The authors validate the dataset with multi-faceted analyses, ablations, and cross-model fine-tuning, showing robustness and improved performance as model scale increases. They demonstrate practical value in persona-based dialogue, where PCoKG yields more consistent and context-aware responses, highlighting its potential for personalized AI systems.

Abstract

Most commonsense reasoning models overlook the influence of personality traits, limiting their effectiveness in personalized systems such as dialogue generation. To address this limitation, we introduce the Personality-aware Commonsense Knowledge Graph (PCoKG), a structured dataset comprising 521,316 quadruples. We begin by employing three evaluators to score and filter events from the ATOMIC dataset, selecting those that are likely to elicit diverse reasoning patterns across different personality types. For knowledge graph construction, we leverage the role-playing capabilities of large language models (LLMs) to perform reasoning tasks. To enhance the quality of the generated knowledge, we incorporate a debate mechanism consisting of a proponent, an opponent, and a judge, which iteratively refines the outputs through feedback loops. We evaluate the dataset from multiple perspectives and conduct fine-tuning and ablation experiments using multiple LLM backbones to assess PCoKG's robustness and the effectiveness of its construction pipeline. Our LoRA-based fine-tuning results indicate a positive correlation between model performance and the parameter scale of the base models. Finally, we apply PCoKG to persona-based dialogue generation, where it demonstrates improved consistency between generated responses and reference outputs. This work bridges the gap between commonsense reasoning and individual cognitive differences, enabling the development of more personalized and context-aware AI systems.
Paper Structure (22 sections, 4 figures, 7 tables, 2 algorithms)

This paper contains 22 sections, 4 figures, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Commonsense Knowledge Bases
  4. Myers-Briggs Type Indicator
  5. Construction of the Personality-aware Commonsense Knowledge Graph
  6. Event and Reasoning Dimension Acquisition
  7. Personality-aware Reasoning via Debate
  8. Dataset Analysis
  9. Readability-Personality Association Analysis
  10. Personality-Reasoning Association Analysis
  11. Human Evaluation
  12. Experiments
  13. Setup
  14. Dataset
  15. Metrics
  16. ...and 7 more sections

Figures (4)

  • Figure 1: The upper panel shows how COMET is commonly used, while the lower panel illustrates a more realistic application incorporating personal traits.
  • Figure 2: PCoKG construction process.
  • Figure 3: Flesch reading ease scores across 16 MBTI personality types.
  • Figure 4: Performance of the foundation model with varying sizes on the PCoKG under LoRA fine-tuning.