Do Zombies Understand? A Choose-Your-Own-Adventure Exploration of Machine Cognition
Ariel Goldstein, Gabriel Stanovsky
TL;DR
The paper addresses the disagreement over whether language models truly understand text by framing a thought experiment around an open-source chatbot $Z$ and two working definitions: Functional Understanding, based on external task performance, and Conscious Understanding, which requires subjective experience. It surveys historical benchmarks and philosophical arguments (e.g., Turing, Chinese Room, NCC/IIT) to show how consciousness can shape the criteria for understanding. The authors clarify terminological gaps, propose parallel research agendas, and discuss how neuroscience insights could inform AI design, ultimately guiding future evaluation of machine cognition. This framework helps bridge AI benchmarks with cognitive science to refine how we assess and interpret machine cognition in practice.
Abstract
Recent advances in LLMs have sparked a debate on whether they understand text. In this position paper, we argue that opponents in this debate hold different definitions for understanding, and particularly differ in their view on the role of consciousness. To substantiate this claim, we propose a thought experiment involving an open-source chatbot $Z$ which excels on every possible benchmark, seemingly without subjective experience. We ask whether $Z$ is capable of understanding, and show that different schools of thought within seminal AI research seem to answer this question differently, uncovering their terminological disagreement. Moving forward, we propose two distinct working definitions for understanding which explicitly acknowledge the question of consciousness, and draw connections with a rich literature in philosophy, psychology and neuroscience.