Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CogGPT: Unleashing the Power of Cognitive Dynamics on Large Language Models

Yaojia Lv, Haojie Pan, Zekun Wang, Jiafeng Liang, Yuanxing Liu, Ruiji Fu, Ming Liu, Zhongyuan Wang, Bing Qin

TL;DR

The paper argues that current LLM studies treat cognition as static and fail to capture lifelong cognitive dynamics. It introduces CogBench, a 22,000-instance benchmark with multi-source information flows (across article- and video-based streams) and two metrics—Authenticity and Rationality—to assess how well LLM-driven agents adapt their cognitive state over iterations. The authors then present CogGPT, an LLM-driven agent with a memory retention system and a collaborative refinement framework that enable iterative, memory-guided updates to its profile and reasoning as information arrives. Empirical results show CogGPT outperforms static baselines (CoT, ReAct, Reflexion) in both alignment of ratings and quality of reasoning, under both article- and video-based information flows, with substantial human-evaluation agreement. The work advances the study of cognitive dynamics in LLMs and points to future directions for human-in-the-loop sandbox environments to better understand and leverage lifelong cognitive adaptation in AI systems.

Abstract

Cognitive dynamics are pivotal to advance human understanding of the world. Recent advancements in large language models (LLMs) reveal their potential for cognitive simulation. However, these LLM-based cognitive studies primarily focus on static modeling, overlooking the dynamic nature of cognition. To bridge this gap, we propose the concept of the cognitive dynamics of LLMs and present a corresponding task with the inspiration of longitudinal studies. Towards the task, we develop CogBench, a novel benchmark to assess the cognitive dynamics of LLMs and validate it through participant surveys. We also design two evaluation metrics for CogBench, including Authenticity and Rationality. Recognizing the inherent static nature of LLMs, we introduce CogGPT for the task, which features an innovative iterative cognitive mechanism aimed at enhancing lifelong cognitive dynamics. Empirical results demonstrate the superiority of CogGPT over existing methods, particularly in its ability to facilitate role-specific cognitive dynamics under continuous information flows.

CogGPT: Unleashing the Power of Cognitive Dynamics on Large Language Models

TL;DR

The paper argues that current LLM studies treat cognition as static and fail to capture lifelong cognitive dynamics. It introduces CogBench, a 22,000-instance benchmark with multi-source information flows (across article- and video-based streams) and two metrics—Authenticity and Rationality—to assess how well LLM-driven agents adapt their cognitive state over iterations. The authors then present CogGPT, an LLM-driven agent with a memory retention system and a collaborative refinement framework that enable iterative, memory-guided updates to its profile and reasoning as information arrives. Empirical results show CogGPT outperforms static baselines (CoT, ReAct, Reflexion) in both alignment of ratings and quality of reasoning, under both article- and video-based information flows, with substantial human-evaluation agreement. The work advances the study of cognitive dynamics in LLMs and points to future directions for human-in-the-loop sandbox environments to better understand and leverage lifelong cognitive adaptation in AI systems.

Abstract

Cognitive dynamics are pivotal to advance human understanding of the world. Recent advancements in large language models (LLMs) reveal their potential for cognitive simulation. However, these LLM-based cognitive studies primarily focus on static modeling, overlooking the dynamic nature of cognition. To bridge this gap, we propose the concept of the cognitive dynamics of LLMs and present a corresponding task with the inspiration of longitudinal studies. Towards the task, we develop CogBench, a novel benchmark to assess the cognitive dynamics of LLMs and validate it through participant surveys. We also design two evaluation metrics for CogBench, including Authenticity and Rationality. Recognizing the inherent static nature of LLMs, we introduce CogGPT for the task, which features an innovative iterative cognitive mechanism aimed at enhancing lifelong cognitive dynamics. Empirical results demonstrate the superiority of CogGPT over existing methods, particularly in its ability to facilitate role-specific cognitive dynamics under continuous information flows.
Paper Structure (32 sections, 2 equations, 7 figures, 6 tables)

This paper contains 32 sections, 2 equations, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Task Definition
  3. CogBench
  4. Data Construction
  5. Data Validation
  6. Evaluation Metrics
  7. Method
  8. Memory Retention System
  9. Collaborative Refinement Framework
  10. Experiments
  11. Experimental Setup
  12. Evaluation Results
  13. Influence of Different Information Flows
  14. Human Evaluation Agreement
  15. Case Study
  16. ...and 17 more sections

Figures (7)

  • Figure 1: A case of human cognitive dynamics. A man (on the left) undergoes a gradual shift in his perspective of the universe, influenced by continuous information flows (on the right).
  • Figure 2: An example of human cognitive dynamics in response to the same question in both CogBench-v and CogBench-a. The continuous changes in human ratings significantly validate the effectiveness of CogBench.
  • Figure 3: Overview of the architecture of CogGPT. CogGPT incorporates a novel iterative cognitive mechanism, comprising two crucial components: a memory retention system for continuous information perception, and a collaborative refinement framework designed for lifelong cognitive dynamics.
  • Figure 4: Comparative analysis of CogGPT's performance in CogBench-v and CogBench-a. Panel (a) showcases the average Authenticity scores, and Panel (b) presents the average Rationality scores. These results highlight the consistent impact of different information flows on the cognitive dynamics of LLMs.
  • Figure 5: Comparative analysis of different agents in assessing the psychological risks of outdoor adventures. CoT, ReAct and Reflexion utilize an initial profile and current information flow due to their static cognitive framework. In contrast, CogGPT benefits from its iterative cognitive mechanism, enabling a dynamic profile and real-time memory recall. yellow highlights represent clues from profiles, while blue highlights indicate clues from memory. Green highlights denote appropriate responses, and red highlights signify inappropriate responses. This comparison demonstrates that CogGPT exhibits closer alignment with human expectations in both rating and reasoning.
  • ...and 2 more figures