Signaling and Social Learning in Swarms of Robots

TL;DR

A taxonomy of existing and future works on communication, focusing on information selection and physical abstraction as principal axes for classification, is proposed: from low-level lossless compression with raw signal extraction and processing to high-level lossy compression with structured communication models.

Abstract

This paper investigates the role of communication in improving coordination within robot swarms, focusing on a paradigm where learning and execution occur simultaneously in a decentralized manner. We highlight the role communication can play in addressing the credit assignment problem (individual contribution to the overall performance), and how it can be influenced by it. We propose a taxonomy of existing and future works on communication, focusing on information selection and physical abstraction as principal axes for classification: from low-level lossless compression with raw signal extraction and processing to high-level lossy compression with structured communication models. The paper reviews current research from evolutionary robotics, multi-agent (deep) reinforcement learning, language models, and biophysics models to outline the challenges and opportunities of communication in a collective of robots that continuously learn from one another through local message exchanges, illustrating a form of social learning.

Figures (3)

• Figure 1: (A) A swarm of robots is deployed in an unknown environment. Robots must learn together to solve a task. Robots interact locally with nearby robots and physical elements. (B) The decision-making process of a focal robot is based on cues from the physical world and signals from the social world. (C) Diagram of the communication and control policies for a robot, distinguishing between signals for local interactions and cues from the broader environment. The pink box denotes the policy of the robot which gets information from observations (i.e., cues and signals) and produces actions (i.e., effectors and communication channels). There are two sub-policies for each process, though in practice a single general policy may be used (e.g., a single artificial neural network), or multiple policies, either ad hoc or subject to learning.
• Figure 2: Alignment of Nash Equilibrium with Social Welfare with respect to the degree of inclusive fitness and the degree of shared interest among robots. The X-axis shows how aligned the individual’s interest (e.g., its local fitness function) is with that of the group, which is uniquely defined by its ability to optimally solve the task. The Y-axis shows the level of inclusive fitness experienced by each individual in the population (e.g., due to kin recognition, environmental viscosity, etc.). The four text boxes on the graph provide examples using the well-known theoretical games of Prisoner’s Dilemma (PD, a competitive game where players should defect) and Stag Hunt (SH, a coordination game where players should cooperate), and two extremes regarding how inclusive is an individual’s fitness in a population (unrelated individuals working for their own sake vs. a population of clones working for the collective).
• Figure 3: Signaling methods can be projected in a two-dimensional plane using information selection and physical abstraction as main components. Left: An algebraic analogy for information selection and physical abstraction in communication processes. Changing the degree of information selection can be done through operations like restriction to a subspace (projection with or without loss). Changing the level of physical abstraction can be done via a change of basis, such as transforming a complex matrix into a simpler diagonal form, illustrating how information can be simplified and structured. Right: Different approaches to communication in robotics, mapped by information selection and physical abstraction. Low-level methods include biophysics-inspired processes, while high-level methods involve language models and emergent languages.