Cognitive Architecture Toward Common Ground Sharing Among Humans and Generative AIs: Trial on Model-Model Interactions in Tangram Naming Task

TL;DR

This work addresses the need for transparent common-grounding between humans and generative AIs by developing a modular cognitive-architecture that simulates sender–receiver interactions in the Tangram Naming Task (TNT). It integrates vision-language modules (CNN-based perception, img2img image generation, and captioning) with an imaginal-memory–driven core and a learning loop that updates partner-specific parameters via policy-gradient backpropagation. Initial results show above-chance performance in one-shot communication and statistically significant gains through iterative learning, though current performance still falls short of human TNT alignment, highlighting both the potential and the limitations of the approach. The study contributes a concrete, interpretable framework for exploring common-ground formation and provides a path toward more trustworthy, human-aligned multi-agent AI systems.

Abstract

For generative AIs to be trustworthy, establishing transparent common grounding with humans is essential. As a preparation toward human-model common grounding, this study examines the process of model-model common grounding. In this context, common ground is defined as a cognitive framework shared among agents in communication, enabling the connection of symbols exchanged between agents to the meanings inherent in each agent. This connection is facilitated by a shared cognitive framework among the agents involved. In this research, we focus on the tangram naming task (TNT) as a testbed to examine the common-ground-building process. Unlike previous models designed for this task, our approach employs generative AIs to visualize the internal processes of the model. In this task, the sender constructs a metaphorical image of an abstract figure within the model and generates a detailed description based on this image. The receiver interprets the generated description from the partner by constructing another image and reconstructing the original abstract figure. Preliminary results from the study show an improvement in task performance beyond the chance level, indicating the effect of the common cognitive framework implemented in the models. Additionally, we observed that incremental backpropagations leveraging successful communication cases for a component of the model led to a statistically significant increase in performance. These results provide valuable insights into the mechanisms of common grounding made by generative AIs, improving human communication with the evolving intelligent machines in our future society.

Figures (8)

• Figure 1: An example of placement in the tangram naming task.
• Figure 2: Module composition for the tangram naming task.
• Figure 3: Example of images generated by the img2img in Stable Diffusion.
• Figure 4: Example of holistic process episode.
• Figure 5: Learning process. Each step is assumed to be executed by the module presented in Figure 2.