Wanting to be Understood

TL;DR

This work investigates whether intrinsic motivations for mutual understanding—beyond extrinsic rewards—can drive social interaction in multi-agent systems. Using a PCP-based 1D surveillance/testbed, agents are trained with PPO and an LSTM predictor, comparing artificial curiosity to three reciprocal drives: imitation/imitation-by, influence/impressionability, and sub-reaction time anticipation. The results show that reciprocal drives that reward being understood and understanding others yield sustained self–other coordination and can enable cooperation in tasks where only one agent receives external rewards. This suggests a plausible computational account for early intersubjectivity and has implications for designing cooperative AI that can bootstrap joint behavior without explicit task-shared incentives.

Abstract

This paper explores an intrinsic motivation for mutual awareness, hypothesizing that humans possess a fundamental drive to understand and to be understood even in the absence of extrinsic rewards. Through simulations of the perceptual crossing paradigm, we explore the effect of various internal reward functions in reinforcement learning agents. The drive to understand is implemented as an active inference type artificial curiosity reward, whereas the drive to be understood is implemented through intrinsic rewards for imitation, influence/impressionability, and sub-reaction time anticipation of the other. Results indicate that while artificial curiosity alone does not lead to a preference for social interaction, rewards emphasizing reciprocal understanding successfully drive agents to prioritize interaction. We demonstrate that this intrinsic motivation can facilitate cooperation in tasks where only one agent receives extrinsic reward for the behaviour of the other.

Figures (16)

• Figure 1: The perceptual crossing paradigm, Figure from auvray2009perceptual. Two agents (avatars) live on a line. They can cross each other, each other's shadows, or their own private stationary object.
• Figure 2: The artificial curiosity reward rewards policies that minimize errors when certainty is low, but maximizes errors when certainty is high. This is analogous to conducting scientific experiments by setting up a situation in which a prediction is strong, and attempting to contradict that prediction. Reward is shown on the y-axis, and certainty on the x-axis.
• Figure 3: Mutual information is calculated (between two pairs of crossing transitions) when either of two interaction conditions are met. For passive-->active interactions an agent must do two no-ops, then at least one op with an associated crossing transition. For active-->passive interactions an agent must do at least one op associated with a crossing transition followed by two no-ops.
• Figure 4: Mean and standard deviation of rewards and crossing frequencies averaged over both agents over 5 runs.
• Figure 5: Artificial Curiosity Reward: Artificial curiosity does not produce an overall preference for interaction with each other. Agents actively explore both