Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HUMANLLM: Benchmarking and Reinforcing LLM Anthropomorphism via Human Cognitive Patterns

Xintao Wang, Jian Yang, Weiyuan Li, Rui Xie, Jen-tse Huang, Jun Gao, Shuai Huang, Yueping Kang, Liyuan Gou, Hongwei Feng, Yanghua Xiao

TL;DR

HUMANLLM-8B outperforms Qwen3-32B on multi-pattern dynamics despite 4x fewer parameters, demonstrating that authentic anthropomorphism requires cognitive modeling--simulating not just what humans do, but the psychological processes generating those behaviors.

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities in reasoning and generation, serving as the foundation for advanced persona simulation and Role-Playing Language Agents (RPLAs). However, achieving authentic alignment with human cognitive and behavioral patterns remains a critical challenge for these agents. We present HUMANLLM, a framework treating psychological patterns as interacting causal forces. We construct 244 patterns from ~12,000 academic papers and synthesize 11,359 scenarios where 2-5 patterns reinforce, conflict, or modulate each other, with multi-turn conversations expressing inner thoughts, actions, and dialogue. Our dual-level checklists evaluate both individual pattern fidelity and emergent multi-pattern dynamics, achieving strong human alignment (r=0.91) while revealing that holistic metrics conflate simulation accuracy with social desirability. HUMANLLM-8B outperforms Qwen3-32B on multi-pattern dynamics despite 4x fewer parameters, demonstrating that authentic anthropomorphism requires cognitive modeling--simulating not just what humans do, but the psychological processes generating those behaviors.

HUMANLLM: Benchmarking and Reinforcing LLM Anthropomorphism via Human Cognitive Patterns

TL;DR

HUMANLLM-8B outperforms Qwen3-32B on multi-pattern dynamics despite 4x fewer parameters, demonstrating that authentic anthropomorphism requires cognitive modeling--simulating not just what humans do, but the psychological processes generating those behaviors.

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities in reasoning and generation, serving as the foundation for advanced persona simulation and Role-Playing Language Agents (RPLAs). However, achieving authentic alignment with human cognitive and behavioral patterns remains a critical challenge for these agents. We present HUMANLLM, a framework treating psychological patterns as interacting causal forces. We construct 244 patterns from ~12,000 academic papers and synthesize 11,359 scenarios where 2-5 patterns reinforce, conflict, or modulate each other, with multi-turn conversations expressing inner thoughts, actions, and dialogue. Our dual-level checklists evaluate both individual pattern fidelity and emergent multi-pattern dynamics, achieving strong human alignment (r=0.91) while revealing that holistic metrics conflate simulation accuracy with social desirability. HUMANLLM-8B outperforms Qwen3-32B on multi-pattern dynamics despite 4x fewer parameters, demonstrating that authentic anthropomorphism requires cognitive modeling--simulating not just what humans do, but the psychological processes generating those behaviors.
Paper Structure (82 sections, 4 figures, 22 tables)

This paper contains 82 sections, 4 figures, 22 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. HumanLLM Dataset
  4. Pattern Collection
  5. Personality Traits (Person Dimension)
  6. Social-Cognitive Patterns (Environment Dimension)
  7. Pattern Data Construction
  8. Literature Retrieval
  9. Pattern Synthesis
  10. Scenario and Conversation Generation
  11. Scenario Synthesis
  12. Conversation Synthesis
  13. Dual-Level Checklist Design
  14. Scenario-Level Checklist.
  15. Pattern-Level Checklist
  16. ...and 67 more sections

Figures (4)

  • Figure 1: Pattern Data Structure: 144 Social-Cognitive Patterns (left) and 100 Personality Traits (right). Each pattern comprises Definition, Core Mechanisms, and Real-World Manifestations
  • Figure 2: HumanLLM Framework.Left: Dataset structure with scenarios, multi-turn conversations (inner thoughts in brackets, actions in parentheses), and dual-level checklists. Top Right: Supervised fine-tuning on target character utterances. Bottom Right: Evaluation via LLM judge scoring against pattern-level and scenario-level checklists.
  • Figure 3: Human-LLM evaluation alignment comparison. Left: Holistic metrics show clear separation between LLM and human judgments, with systematic LLM underestimation of psychologically complex behaviors. Right: Our checklist metrics demonstrate strong overlap between LLM and human distributions, indicating robust alignment with expert judgment
  • Figure 4: Dataset distributions: (a) number of dialogue turns per conversation (range: 12--20, mean: 16.4); (b) number of patterns per scenario (range: 2--5, mean: 3.5).