Toward Cognitive and Immersive Systems: Experiments in a Cognitive Microworld

TL;DR

This work defines cognitive and immersive systems (CAIS) that reason about human mental states to assist group problem-solving. It grounds CAIS in a formal framework built on the Deontic Cognitive Event Calculus ($\mathcal{DCEC}$) and the Cognitive Polysolid Framework (CPF), enabling rich theory-of-mind reasoning and planning. The authors implement reasoning and planning components (ShadowProver and Spectra) and demonstrate a cognitive polysolid world simulation with false-belief tasks to show CAIS usefulness in coordinating and correcting human beliefs. The contribution advances principled CAIS design, offers a formalizable set of requirements, and outlines avenues for extending cognitive domains and social-choice considerations in future work.

Abstract

As computational power has continued to increase, and sensors have become more accurate, the corresponding advent of systems that are at once cognitive and immersive has arrived. These \textit{cognitive and immersive systems} (CAISs) fall squarely into the intersection of AI with HCI/HRI: such systems interact with and assist the human agents that enter them, in no small part because such systems are infused with AI able to understand and reason about these humans and their knowledge, beliefs, goals, communications, plans, etc. We herein explain our approach to engineering CAISs. We emphasize the capacity of a CAIS to develop and reason over a `theory of the mind' of its human partners. This capacity entails that the AI in question has a sophisticated model of the beliefs, knowledge, goals, desires, emotions, etc.\ of these humans. To accomplish this engineering, a formal framework of very high expressivity is needed. In our case, this framework is a \textit{cognitive event calculus}, a particular kind of quantified multi-operator modal logic, and a matching high-expressivity automated reasoner and planner. To explain, advance, and to a degree validate our approach, we show that a calculus of this type satisfies a set of formal requirements, and can enable a CAIS to understand a psychologically tricky scenario couched in what we call the \textit{cognitive polysolid framework} (CPF). We also formally show that a room that satisfies these requirements can have a useful property we term \emph{expectation of usefulness}. CPF, a sub-class of \textit{cognitive microworlds}, includes machinery able to represent and plan over not merely blocks and actions (such as seen in the primitive `blocks worlds' of old), but also over agents and their mental attitudes about both other agents and inanimate objects.

Figures (2)

• Figure 1: The flow of information through the areas of a CAIS.
• Figure 2: Visual representation of mental states of the agents ( $\hbox{$h_2$}$'s state is grayed as he is not in the room).