Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Dialogue Game for Eliciting Balanced Collaboration

Isidora Jeknić, David Schlangen, Alexander Koller

TL;DR

Addresses the need for balanced collaboration in dialogue by moving beyond fixed instructor/follower roles. It introduces a two-player online 2D object-placement game where the target state must be negotiated via chat and is not predetermined. The study defines a dominance score using $RD = (\mathrm{volume}_A - \mathrm{volume}_B)/(\mathrm{volume}_A + \mathrm{volume}_B)$ and $D_A = \mathrm{verbosity}_A \cdot L(RD)$ with $L(x) = 1/(1+e^{-x})$, analyzes four collaboration strategies, and finds balanced back-and-forth interactions yield higher task performance than Leader-dominated play. It also presents a baseline LLM agent that can participate in the game but remains below human performance, underscoring the challenge of human-AI collaboration in balanced settings and motivating further research.

Abstract

Collaboration is an integral part of human dialogue. Typical task-oriented dialogue games assign asymmetric roles to the participants, which limits their ability to elicit naturalistic role-taking in collaboration and its negotiation. We present a novel and simple online setup that favors balanced collaboration: a two-player 2D object placement game in which the players must negotiate the goal state themselves. We show empirically that human players exhibit a variety of role distributions, and that balanced collaboration improves task performance. We also present an LLM-based baseline agent which demonstrates that automatic playing of our game is an interesting challenge for artificial systems.

A Dialogue Game for Eliciting Balanced Collaboration

TL;DR

Addresses the need for balanced collaboration in dialogue by moving beyond fixed instructor/follower roles. It introduces a two-player online 2D object-placement game where the target state must be negotiated via chat and is not predetermined. The study defines a dominance score using and with , analyzes four collaboration strategies, and finds balanced back-and-forth interactions yield higher task performance than Leader-dominated play. It also presents a baseline LLM agent that can participate in the game but remains below human performance, underscoring the challenge of human-AI collaboration in balanced settings and motivating further research.

Abstract

Collaboration is an integral part of human dialogue. Typical task-oriented dialogue games assign asymmetric roles to the participants, which limits their ability to elicit naturalistic role-taking in collaboration and its negotiation. We present a novel and simple online setup that favors balanced collaboration: a two-player 2D object placement game in which the players must negotiate the goal state themselves. We show empirically that human players exhibit a variety of role distributions, and that balanced collaboration improves task performance. We also present an LLM-based baseline agent which demonstrates that automatic playing of our game is an interesting challenge for artificial systems.
Paper Structure (13 sections, 1 equation, 7 figures, 4 tables)

This paper contains 13 sections, 1 equation, 7 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Collaborative object-placement game
  4. Game playing strategies
  5. Collaboration strategies
  6. Dominance scores
  7. Impact of strategy on task success
  8. Baseline LLM agent
  9. Conclusion
  10. Appendix
  11. Game environment design.
  12. Baseline agent.
  13. Strategy examples

Figures (7)

  • Figure 1: A reconstructed task view of both players illustrating the shared information (middle, chat box) and information only available to each respective player (left and right). Additionally, illustrates an instance of the back and forth strategy (as described in Section \ref{['sec:strats']}).
  • Figure 2: Overview of strategies; left graph shows the mean scores in each round for each strategy (out of 100), while the right graph shows the distribution of bonuses (score > 99) per strategy in each round (expressed in %).
  • Figure 3: The background images for the two rounds.
  • Figure 4: Leader strategy example
  • Figure 5: Back and forth strategy example
  • ...and 2 more figures