Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Semantics and Spatiality of Emergent Communication

Rotem Ben Zion, Boaz Carmeli, Orr Paradise, Yonatan Belinkov

TL;DR

It is proved, under mild assumptions, that semantically inconsistent communication protocols can be optimal solutions to the discrimination task, but not to reconstruction, and it is shown that the reconstruction objective encourages a stricter property, spatial meaningfulness, which also accounts for the distance between messages.

Abstract

When artificial agents are jointly trained to perform collaborative tasks using a communication channel, they develop opaque goal-oriented communication protocols. Good task performance is often considered sufficient evidence that meaningful communication is taking place, but existing empirical results show that communication strategies induced by common objectives can be counterintuitive whilst solving the task nearly perfectly. In this work, we identify a goal-agnostic prerequisite to meaningful communication, which we term semantic consistency, based on the idea that messages should have similar meanings across instances. We provide a formal definition for this idea, and use it to compare the two most common objectives in the field of emergent communication: discrimination and reconstruction. We prove, under mild assumptions, that semantically inconsistent communication protocols can be optimal solutions to the discrimination task, but not to reconstruction. We further show that the reconstruction objective encourages a stricter property, spatial meaningfulness, which also accounts for the distance between messages. Experiments with emergent communication games validate our theoretical results. These findings demonstrate an inherent advantage of distance-based communication goals, and contextualize previous empirical discoveries.

Semantics and Spatiality of Emergent Communication

TL;DR

It is proved, under mild assumptions, that semantically inconsistent communication protocols can be optimal solutions to the discrimination task, but not to reconstruction, and it is shown that the reconstruction objective encourages a stricter property, spatial meaningfulness, which also accounts for the distance between messages.

Abstract

When artificial agents are jointly trained to perform collaborative tasks using a communication channel, they develop opaque goal-oriented communication protocols. Good task performance is often considered sufficient evidence that meaningful communication is taking place, but existing empirical results show that communication strategies induced by common objectives can be counterintuitive whilst solving the task nearly perfectly. In this work, we identify a goal-agnostic prerequisite to meaningful communication, which we term semantic consistency, based on the idea that messages should have similar meanings across instances. We provide a formal definition for this idea, and use it to compare the two most common objectives in the field of emergent communication: discrimination and reconstruction. We prove, under mild assumptions, that semantically inconsistent communication protocols can be optimal solutions to the discrimination task, but not to reconstruction. We further show that the reconstruction objective encourages a stricter property, spatial meaningfulness, which also accounts for the distance between messages. Experiments with emergent communication games validate our theoretical results. These findings demonstrate an inherent advantage of distance-based communication goals, and contextualize previous empirical discoveries.

Paper Structure

This paper contains 55 sections, 10 theorems, 91 equations, 10 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and related work
  3. Emergent communication with Lewis games
  4. Towards interpretable human-like communication
  5. Input information encoded in the message
  6. Notation and task definitions
  7. General notation
  8. Task definitions
  9. Reconstruction.
  10. Discrimination.
  11. Semantic consistency: a prerequisite to meaningful communication
  12. Are EC systems semantically consistent?
  13. Reconstruction results
  14. Discrimination results
  15. A missing dimension: spatiality in the message space
  16. ...and 40 more sections

Key Result

Lemma 5.1

[proof in page proof:reco_objective] Let $\left(\mathcal{X}, {f_{X}}, M, \Theta, \varPhi, \ell\right)$ be a reconstruction game. Assuming $\varPhi$ is unrestricted, a sender $S_\theta$ is optimal if and only if it minimizes the following objective:

Figures (10)

  • Figure 1: Illustration of the reconstruction and discrimination tasks. The discrimination receiver is given the candidates in addition to the message.
  • Figure 2: Notation for the emergent communication (EC) setup.
  • Figure 3: A message describes a set of inputs. Note: the shapes and colors are not part of the input.
  • Figure 4: Average message purity, comparing trained models to random baselines.
  • Figure 5: Encoder and Decoder architectures used on Shapes.
  • ...and 5 more figures

Theorems & Definitions (31)

  • Definition 1
  • Definition 2
  • Definition 3
  • Remark
  • Lemma 5.1
  • Theorem 5.2
  • Lemma 5.3
  • Corollary 5.4
  • Theorem 5.5
  • Definition 4
  • ...and 21 more