Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

COKE: A Cognitive Knowledge Graph for Machine Theory of Mind

Jincenzi Wu, Zhuang Chen, Jiawen Deng, Sahand Sabour, Helen Meng, Minlie Huang

TL;DR

This work introduces COKE, the first cognitive knowledge graph for machine Theory of Mind, framing ToM as Situation→Clue→Thought→(Action+Emotion) cognitive chains with positive/negative polarity and enabling both explicit data construction and generalization via COLM, a cognitive language model built on LLaMA-2. The authors collect a large, manually curated dataset (over 62k nodes and 45k cognitive chains) through a two-step process combining LLM generation and human annotation, then train COLM to infer cognitive chains for unseen situations through four generation tasks. Automatic and human evaluations show COLM outperforms strong baselines (including GPT-4) on cognitive generation tasks, and integrating COKE into emotional support conversation yields measurable improvements in empathy and usefulness. The work demonstrates the potential of a ToM-enabled AI to better understand and respond to human mental states, with practical impact on social AI applications, while outlining limitations in data coverage and inference capability for future work.

Abstract

Theory of mind (ToM) refers to humans' ability to understand and infer the desires, beliefs, and intentions of others. The acquisition of ToM plays a key role in humans' social cognition and interpersonal relations. Though indispensable for social intelligence, ToM is still lacking for modern AI and NLP systems since they cannot access the human mental state and cognitive process beneath the training corpus. To empower AI systems with the ToM ability and narrow the gap between them and humans, in this paper, we propose COKE: the first cognitive knowledge graph for machine theory of mind. Specifically, COKE formalizes ToM as a collection of 45k+ manually verified cognitive chains that characterize human mental activities and subsequent behavioral/affective responses when facing specific social circumstances. In addition, we further generalize COKE using LLMs and build a powerful generation model COLM tailored for cognitive reasoning. Experimental results in both automatic and human evaluation demonstrate the high quality of COKE, the superior ToM ability of COLM, and its potential to significantly enhance social applications.

COKE: A Cognitive Knowledge Graph for Machine Theory of Mind

TL;DR

This work introduces COKE, the first cognitive knowledge graph for machine Theory of Mind, framing ToM as Situation→Clue→Thought→(Action+Emotion) cognitive chains with positive/negative polarity and enabling both explicit data construction and generalization via COLM, a cognitive language model built on LLaMA-2. The authors collect a large, manually curated dataset (over 62k nodes and 45k cognitive chains) through a two-step process combining LLM generation and human annotation, then train COLM to infer cognitive chains for unseen situations through four generation tasks. Automatic and human evaluations show COLM outperforms strong baselines (including GPT-4) on cognitive generation tasks, and integrating COKE into emotional support conversation yields measurable improvements in empathy and usefulness. The work demonstrates the potential of a ToM-enabled AI to better understand and respond to human mental states, with practical impact on social AI applications, while outlining limitations in data coverage and inference capability for future work.

Abstract

Theory of mind (ToM) refers to humans' ability to understand and infer the desires, beliefs, and intentions of others. The acquisition of ToM plays a key role in humans' social cognition and interpersonal relations. Though indispensable for social intelligence, ToM is still lacking for modern AI and NLP systems since they cannot access the human mental state and cognitive process beneath the training corpus. To empower AI systems with the ToM ability and narrow the gap between them and humans, in this paper, we propose COKE: the first cognitive knowledge graph for machine theory of mind. Specifically, COKE formalizes ToM as a collection of 45k+ manually verified cognitive chains that characterize human mental activities and subsequent behavioral/affective responses when facing specific social circumstances. In addition, we further generalize COKE using LLMs and build a powerful generation model COLM tailored for cognitive reasoning. Experimental results in both automatic and human evaluation demonstrate the high quality of COKE, the superior ToM ability of COLM, and its potential to significantly enhance social applications.
Paper Structure (36 sections, 13 figures, 13 tables)

This paper contains 36 sections, 13 figures, 13 tables.

Table of Contents

  1. Introduction
  2. COKE: Cognitive Knowledge Graph
  3. Preliminaries
  4. What is Theory of Mind?
  5. Why Do AI Systems Need ToM?
  6. Data Structure of COKE
  7. Data Creation and Selection
  8. Prompting LLMs for Situations
  9. Prompting LLMs for Thoughts
  10. Prompting LLMs for Clues, Actions, and Emotions
  11. Human Selection and Revision
  12. Cognitive Language Model COLM
  13. Cognitive Generation Tasks
  14. Training COLM
  15. Experiments
  16. ...and 21 more sections

Figures (13)

  • Figure 1: COKE instantiates Theory of Mind as positive and negative cognitive chains in social situations. Situation$\Rightarrow$ Clue $\Rightarrow$ Thought$\Rightarrow$ (Action $+$ Emotion).
  • Figure 2: The two-step data collection approach for constructing COKE.
  • Figure 3: Prompting LLMs for situations.
  • Figure 4: Empowering ESC with COLM. A detailed case study is illustrated in Table \ref{['tab-esc-case']}.
  • Figure 5: Instruction for Thought selection and revision.
  • ...and 8 more figures