Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Humanlike Cognitive Patterns as Emergent Phenomena in Large Language Models

Zhisheng Tang, Mayank Kejriwal

TL;DR

The paper tackles whether emergent cognitive patterns in LLMs mirror human decision-making, reasoning, and creativity by systematically reviewing studies that benchmark LLMs against human performance on established tasks. It finds that LLMs exhibit several human-like decision biases, GPT-4 shows more deliberative System-2-like reasoning with improved task comprehension, and language-based creativity is robust while divergent thinking and real-world grounding lag. The authors discuss limitations, including memory, attention, and grounding, and advocate for open-model benchmarking and careful deployment to harness collaborative creativity while mitigating risks. The work provides a framework for future research on memory, attention, and open-source development to better understand and guide AI cognition.

Abstract

Research on emergent patterns in Large Language Models (LLMs) has gained significant traction in both psychology and artificial intelligence, motivating the need for a comprehensive review that offers a synthesis of this complex landscape. In this article, we systematically review LLMs' capabilities across three important cognitive domains: decision-making biases, reasoning, and creativity. We use empirical studies drawing on established psychological tests and compare LLMs' performance to human benchmarks. On decision-making, our synthesis reveals that while LLMs demonstrate several human-like biases, some biases observed in humans are absent, indicating cognitive patterns that only partially align with human decision-making. On reasoning, advanced LLMs like GPT-4 exhibit deliberative reasoning akin to human System-2 thinking, while smaller models fall short of human-level performance. A distinct dichotomy emerges in creativity: while LLMs excel in language-based creative tasks, such as storytelling, they struggle with divergent thinking tasks that require real-world context. Nonetheless, studies suggest that LLMs hold considerable potential as collaborators, augmenting creativity in human-machine problem-solving settings. Discussing key limitations, we also offer guidance for future research in areas such as memory, attention, and open-source model development.

Humanlike Cognitive Patterns as Emergent Phenomena in Large Language Models

TL;DR

The paper tackles whether emergent cognitive patterns in LLMs mirror human decision-making, reasoning, and creativity by systematically reviewing studies that benchmark LLMs against human performance on established tasks. It finds that LLMs exhibit several human-like decision biases, GPT-4 shows more deliberative System-2-like reasoning with improved task comprehension, and language-based creativity is robust while divergent thinking and real-world grounding lag. The authors discuss limitations, including memory, attention, and grounding, and advocate for open-model benchmarking and careful deployment to harness collaborative creativity while mitigating risks. The work provides a framework for future research on memory, attention, and open-source development to better understand and guide AI cognition.

Abstract

Research on emergent patterns in Large Language Models (LLMs) has gained significant traction in both psychology and artificial intelligence, motivating the need for a comprehensive review that offers a synthesis of this complex landscape. In this article, we systematically review LLMs' capabilities across three important cognitive domains: decision-making biases, reasoning, and creativity. We use empirical studies drawing on established psychological tests and compare LLMs' performance to human benchmarks. On decision-making, our synthesis reveals that while LLMs demonstrate several human-like biases, some biases observed in humans are absent, indicating cognitive patterns that only partially align with human decision-making. On reasoning, advanced LLMs like GPT-4 exhibit deliberative reasoning akin to human System-2 thinking, while smaller models fall short of human-level performance. A distinct dichotomy emerges in creativity: while LLMs excel in language-based creative tasks, such as storytelling, they struggle with divergent thinking tasks that require real-world context. Nonetheless, studies suggest that LLMs hold considerable potential as collaborators, augmenting creativity in human-machine problem-solving settings. Discussing key limitations, we also offer guidance for future research in areas such as memory, attention, and open-source model development.

Paper Structure

This paper contains 14 sections, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Decision-Making Cognitive Patterns
  3. Heuristics and biases
  4. Evaluating cognitive heuristics and biases in LLMs
  5. Conclusion
  6. Reasoning Cognitive Patterns
  7. Reasoning in humans
  8. Evaluating reasoning in LLMs
  9. Conclusion
  10. Creativity Cognitive Patterns
  11. Human creativity
  12. Creativity in LLMs
  13. Conclusion
  14. Discussion

Figures (1)

  • Figure 1: Timeline of various studies reviewed, along with release time of major Large Language Models. Red, Blue, Green, and Purple entries indicate LLM releases, decision-making related studies, reasoning related studies, and creativity related studies, respectively.