Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-agent Communication meets Natural Language: Synergies between Functional and Structural Language Learning

Angeliki Lazaridou, Anna Potapenko, Olivier Tieleman

TL;DR

This work advances a hybrid approach to language learning by combining a generic language prior with task-grounded multi-agent self-play to produce task-conditioned natural language. By factorizing language use into content planning (what to say) and surface realization (how to say it), the authors enable functional learning guided by task rewards while preserving linguistic structure. They introduce reward-learned rerankers to steer generation without eroding the base language model, and provide a taxonomy and automatic measures for detecting language drift, including pragmatic drift. Empirical results in a visual referential game show that reranker-based methods achieve strong performance with human listeners and mitigate language drift, though challenges remain in aligning agent-driven conventions with human expectations. The work lays groundwork for integrating human-facing natural language with grounded, task-specific communication, while identifying limitations and directions for future research in realistic user interaction and human rewards.

Abstract

We present a method for combining multi-agent communication and traditional data-driven approaches to natural language learning, with an end goal of teaching agents to communicate with humans in natural language. Our starting point is a language model that has been trained on generic, not task-specific language data. We then place this model in a multi-agent self-play environment that generates task-specific rewards used to adapt or modulate the model, turning it into a task-conditional language model. We introduce a new way for combining the two types of learning based on the idea of reranking language model samples, and show that this method outperforms others in communicating with humans in a visual referential communication task. Finally, we present a taxonomy of different types of language drift that can occur alongside a set of measures to detect them.

Multi-agent Communication meets Natural Language: Synergies between Functional and Structural Language Learning

TL;DR

This work advances a hybrid approach to language learning by combining a generic language prior with task-grounded multi-agent self-play to produce task-conditioned natural language. By factorizing language use into content planning (what to say) and surface realization (how to say it), the authors enable functional learning guided by task rewards while preserving linguistic structure. They introduce reward-learned rerankers to steer generation without eroding the base language model, and provide a taxonomy and automatic measures for detecting language drift, including pragmatic drift. Empirical results in a visual referential game show that reranker-based methods achieve strong performance with human listeners and mitigate language drift, though challenges remain in aligning agent-driven conventions with human expectations. The work lays groundwork for integrating human-facing natural language with grounded, task-specific communication, while identifying limitations and directions for future research in realistic user interaction and human rewards.

Abstract

We present a method for combining multi-agent communication and traditional data-driven approaches to natural language learning, with an end goal of teaching agents to communicate with humans in natural language. Our starting point is a language model that has been trained on generic, not task-specific language data. We then place this model in a multi-agent self-play environment that generates task-specific rewards used to adapt or modulate the model, turning it into a task-conditional language model. We introduce a new way for combining the two types of learning based on the idea of reranking language model samples, and show that this method outperforms others in communicating with humans in a visual referential communication task. Finally, we present a taxonomy of different types of language drift that can occur alongside a set of measures to detect them.

Paper Structure

This paper contains 29 sections, 1 figure, 8 tables.

Table of Contents

  1. Introduction
  2. Research framing
  3. Experimental setup
  4. Visual referential communication game.
  5. Dataset and referential splits.
  6. Human performance and setup validation.
  7. Multi-agent communication setup
  8. Speaker
  9. Listener
  10. Methods for learning language use
  11. Functional-only learning
  12. Structural-only learning
  13. Structural and functional learning
  14. Reward finetuning
  15. Multi-task learning
  16. ...and 14 more sections

Figures (1)

  • Figure 1: Example image and ground-truth captions from the Abstract Scenes dataset used in this study.