Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PACE: Abstractions for Communicating Efficiently

Jonathan D. Thomas, Andrea Silvi, Devdatt Dubhashi, Moa Johansson

TL;DR

PACE investigates how procedural abstractions arise under the pressure for efficient communication in collaborative AI tasks. It unites library learning, emergent communication, and reinforcement learning in a neuro-symbolic Architect-Builder framework, employing a contextual bandit to navigate program choices and a MDL-based criterion to introduce abstractions. The results show emergence of concise, human-like language and a stable language after progressive abstraction, with abstractions adopted based on learnability and frequency in the data. This work provides a mechanistic bridge between procedural abstraction learning and efficient communication, offering a path toward human-like conversational abstractions in AI systems.

Abstract

A central but unresolved aspect of problem-solving in AI is the capability to introduce and use abstractions, something humans excel at. Work in cognitive science has demonstrated that humans tend towards higher levels of abstraction when engaged in collaborative task-oriented communication, enabling gradually shorter and more information-efficient utterances. Several computational methods have attempted to replicate this phenomenon, but all make unrealistic simplifying assumptions about how abstractions are introduced and learned. Our method, Procedural Abstractions for Communicating Efficiently (PACE), overcomes these limitations through a neuro-symbolic approach. On the symbolic side, we draw on work from library learning for proposing abstractions. We combine this with neural methods for communication and reinforcement learning, via a novel use of bandit algorithms for controlling the exploration and exploitation trade-off in introducing new abstractions. PACE exhibits similar tendencies to humans on a collaborative construction task from the cognitive science literature, where one agent (the architect) instructs the other (the builder) to reconstruct a scene of block-buildings. PACE results in the emergence of an efficient language as a by-product of collaborative communication. Beyond providing mechanistic insights into human communication, our work serves as a first step to providing conversational agents with the ability for human-like communicative abstractions.

PACE: Abstractions for Communicating Efficiently

TL;DR

PACE investigates how procedural abstractions arise under the pressure for efficient communication in collaborative AI tasks. It unites library learning, emergent communication, and reinforcement learning in a neuro-symbolic Architect-Builder framework, employing a contextual bandit to navigate program choices and a MDL-based criterion to introduce abstractions. The results show emergence of concise, human-like language and a stable language after progressive abstraction, with abstractions adopted based on learnability and frequency in the data. This work provides a mechanistic bridge between procedural abstraction learning and efficient communication, offering a path toward human-like conversational abstractions in AI systems.

Abstract

A central but unresolved aspect of problem-solving in AI is the capability to introduce and use abstractions, something humans excel at. Work in cognitive science has demonstrated that humans tend towards higher levels of abstraction when engaged in collaborative task-oriented communication, enabling gradually shorter and more information-efficient utterances. Several computational methods have attempted to replicate this phenomenon, but all make unrealistic simplifying assumptions about how abstractions are introduced and learned. Our method, Procedural Abstractions for Communicating Efficiently (PACE), overcomes these limitations through a neuro-symbolic approach. On the symbolic side, we draw on work from library learning for proposing abstractions. We combine this with neural methods for communication and reinforcement learning, via a novel use of bandit algorithms for controlling the exploration and exploitation trade-off in introducing new abstractions. PACE exhibits similar tendencies to humans on a collaborative construction task from the cognitive science literature, where one agent (the architect) instructs the other (the builder) to reconstruct a scene of block-buildings. PACE results in the emergence of an efficient language as a by-product of collaborative communication. Beyond providing mechanistic insights into human communication, our work serves as a first step to providing conversational agents with the ability for human-like communicative abstractions.
Paper Structure (21 sections, 2 equations, 10 figures, 4 algorithms)

This paper contains 21 sections, 2 equations, 10 figures, 4 algorithms.

Table of Contents

  1. Introduction
  2. Set-up: Architect-Builder Game
  3. Dataset
  4. The agents
  5. PACE: Selection, Communication and Abstraction
  6. Selecting programs
  7. Communicating a program
  8. Introducing Abstractions
  9. Efficient Communication in PACE
  10. Implementation Details
  11. Results & Discussion
  12. PACE reduces average morphosyntactic complexity
  13. Why does PACE converge?
  14. Which Abstractions are Adopted?
  15. Related Literature
  16. ...and 6 more sections

Figures (10)

  • Figure 1: Two artificial agents playing the architect-builder game, starting from a small artificial language. Initially, the architect messages refer to horizontal or vertical blocks (a). After multiple interactions, the architect tries to introduce an abstraction (b), which after a learning period allows for shorter communication to solve the task (c).
  • Figure 2: Interaction between the architect and builder in PACE proceeds as follows: (1) given a goal-scene the architect chooses the program to communicate, and (2) the architect and builder communicate via EC. After multiple interactions, an abstraction is introduced. Then the loop repeats.
  • Figure 3: Comparison between PACE, Greedy and No abstractions in terms of program length and test reward over time. Line indicates mean value and shaded regions indicate the 95% confidence interval.
  • Figure 4: Adoption rate of possible abstractions by language size. As the language grows, fewer new abstractions are adopted. The dashed line represents the interpolation of the (discrete) Pareto Frontier calculated as the trade-off between average program length and size of the language $\mathcal{A}$. The grey area represents unachievable languages.
  • Figure 5: Different abstractions being introduced. We show the mean and 95% confidence interval for Q-value (black) and frequency (blue) versus epochs (of which there are $40$ in a step).
  • ...and 5 more figures