Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Challenges in Human-Agent Communication

Gagan Bansal, Jennifer Wortman Vaughan, Saleema Amershi, Eric Horvitz, Adam Fourney, Hussein Mozannar, Victor Dibia, Daniel S. Weld

TL;DR

This work frames human–agent communication around common ground, grounded in cognitive science and HCI, to address twelve challenges arising from modern, tool-using base models. It classifies these into three broad families (A1–A5, U1–U3, X1–X4) and provides concrete examples across pre-, during-, and post-action phases, illustrating both goals-to-be-achieved and capabilities-to-be-disclosed. The contribution is a structured taxonomy of information flow challenges, with analysis of corresponding design directions and open research questions, emphasizing transparency, controllability, and iterative verification. The paper advocates cross-disciplinary effort to develop patterns, guidelines, and interfaces that sustain shared understanding in high-stakes, multi-agent workflows, ultimately improving safety and user trust in autonomous systems.

Abstract

Remarkable advancements in modern generative foundation models have enabled the development of sophisticated and highly capable autonomous agents that can observe their environment, invoke tools, and communicate with other agents to solve problems. Although such agents can communicate with users through natural language, their complexity and wide-ranging failure modes present novel challenges for human-AI interaction. Building on prior research and informed by a communication grounding perspective, we contribute to the study of \emph{human-agent communication} by identifying and analyzing twelve key communication challenges that these systems pose. These include challenges in conveying information from the agent to the user, challenges in enabling the user to convey information to the agent, and overarching challenges that need to be considered across all human-agent communication. We illustrate each challenge through concrete examples and identify open directions of research. Our findings provide insights into critical gaps in human-agent communication research and serve as an urgent call for new design patterns, principles, and guidelines to support transparency and control in these systems.

Challenges in Human-Agent Communication

TL;DR

This work frames human–agent communication around common ground, grounded in cognitive science and HCI, to address twelve challenges arising from modern, tool-using base models. It classifies these into three broad families (A1–A5, U1–U3, X1–X4) and provides concrete examples across pre-, during-, and post-action phases, illustrating both goals-to-be-achieved and capabilities-to-be-disclosed. The contribution is a structured taxonomy of information flow challenges, with analysis of corresponding design directions and open research questions, emphasizing transparency, controllability, and iterative verification. The paper advocates cross-disciplinary effort to develop patterns, guidelines, and interfaces that sustain shared understanding in high-stakes, multi-agent workflows, ultimately improving safety and user trust in autonomous systems.

Abstract

Remarkable advancements in modern generative foundation models have enabled the development of sophisticated and highly capable autonomous agents that can observe their environment, invoke tools, and communicate with other agents to solve problems. Although such agents can communicate with users through natural language, their complexity and wide-ranging failure modes present novel challenges for human-AI interaction. Building on prior research and informed by a communication grounding perspective, we contribute to the study of \emph{human-agent communication} by identifying and analyzing twelve key communication challenges that these systems pose. These include challenges in conveying information from the agent to the user, challenges in enabling the user to convey information to the agent, and overarching challenges that need to be considered across all human-agent communication. We illustrate each challenge through concrete examples and identify open directions of research. Our findings provide insights into critical gaps in human-agent communication research and serve as an urgent call for new design patterns, principles, and guidelines to support transparency and control in these systems.

Paper Structure

This paper contains 24 sections, 1 figure.

Table of Contents

  1. Introduction
  2. What is different about modern AI agents?
  3. Outline
  4. Previous Work
  5. Overarching Challenges for Human-Agent Communication
  6. Information Flow from the User to the Agent
  7. Example 1: Travel Itinerary Planning
  8. Example 2: Scientific Literature Search
  9. Example 1: Privacy Concerns in Data Sharing
  10. Example 2: Web Scraping with Ethical Considerations
  11. Example 1: Code Debugging
  12. Example: Travel Itinerary Adjustments
  13. Information Flow from the Agent to the User
  14. Example 1: Dataset Visualization
  15. Example 2: Proving Theorems
  16. ...and 9 more sections

Figures (1)

  • Figure 1: Modern agents introduce new (or newly complex) challenges for human-agent communication. We group these challenges broadly into three categories: those that pertain to information agents need to convey to users (A1--A5), those that pertain to information users need to convey to agents (U1--U3) and general difficulties with human-agent communication (X1--X4). These challenges may be differently applicable across the phases of human-agent interaction (x-axis): before, during, and after an interaction/execution.