Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Establishing Shared Query Understanding in an Open Multi-Agent System

Nikolaos Kondylidis, Ilaria Tiddi, Annette ten Teije

TL;DR

The paper tackles the problem of establishing shared task understanding between two agents in an open multi-agent system, particularly when one agent lacks a formal ontology. It introduces a teacher–student framework with grounded, example-based communication and environment-based evaluation, enabling indirect assessment of understanding through downstream task performance. The authors instantiate the framework in cooperative query answering across heterogeneous ontologies and compare teaching and learning policies under defined communication, cooperation, and efficiency restrictions. Results indicate that robust shared understanding can be achieved with a small number of concise interactions, and the framework supports human-in-the-loop experimentation and broader applicability to OMAS contexts.

Abstract

We propose a method that allows to develop shared understanding between two agents for the purpose of performing a task that requires cooperation. Our method focuses on efficiently establishing successful task-oriented communication in an open multi-agent system, where the agents do not know anything about each other and can only communicate via grounded interaction. The method aims to assist researchers that work on human-machine interaction or scenarios that require a human-in-the-loop, by defining interaction restrictions and efficiency metrics. To that end, we point out the challenges and limitations of such a (diverse) setup, while also restrictions and requirements which aim to ensure that high task performance truthfully reflects the extent to which the agents correctly understand each other. Furthermore, we demonstrate a use-case where our method can be applied for the task of cooperative query answering. We design the experiments by modifying an established ontology alignment benchmark. In this example, the agents want to query each other, while representing different databases, defined in their own ontologies that contain different and incomplete knowledge. Grounded interaction here has the form of examples that consists of common instances, for which the agents are expected to have similar knowledge. Our experiments demonstrate successful communication establishment under the required restrictions, and compare different agent policies that aim to solve the task in an efficient manner.

Establishing Shared Query Understanding in an Open Multi-Agent System

TL;DR

The paper tackles the problem of establishing shared task understanding between two agents in an open multi-agent system, particularly when one agent lacks a formal ontology. It introduces a teacher–student framework with grounded, example-based communication and environment-based evaluation, enabling indirect assessment of understanding through downstream task performance. The authors instantiate the framework in cooperative query answering across heterogeneous ontologies and compare teaching and learning policies under defined communication, cooperation, and efficiency restrictions. Results indicate that robust shared understanding can be achieved with a small number of concise interactions, and the framework supports human-in-the-loop experimentation and broader applicability to OMAS contexts.

Abstract

We propose a method that allows to develop shared understanding between two agents for the purpose of performing a task that requires cooperation. Our method focuses on efficiently establishing successful task-oriented communication in an open multi-agent system, where the agents do not know anything about each other and can only communicate via grounded interaction. The method aims to assist researchers that work on human-machine interaction or scenarios that require a human-in-the-loop, by defining interaction restrictions and efficiency metrics. To that end, we point out the challenges and limitations of such a (diverse) setup, while also restrictions and requirements which aim to ensure that high task performance truthfully reflects the extent to which the agents correctly understand each other. Furthermore, we demonstrate a use-case where our method can be applied for the task of cooperative query answering. We design the experiments by modifying an established ontology alignment benchmark. In this example, the agents want to query each other, while representing different databases, defined in their own ontologies that contain different and incomplete knowledge. Grounded interaction here has the form of examples that consists of common instances, for which the agents are expected to have similar knowledge. Our experiments demonstrate successful communication establishment under the required restrictions, and compare different agent policies that aim to solve the task in an efficient manner.
Paper Structure (43 sections, 1 equation, 5 figures)

This paper contains 43 sections, 1 equation, 5 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Ontology Alignment
  4. Incremental OA Through Interaction
  5. Agent Communication Establishment
  6. Communication Establishment in Open Multi-Agent Systems
  7. Language Games
  8. Interactive Reinforcement Learning
  9. A Framework for Developing and Evaluating Human-Hybrid Understanding
  10. The Process of Defining the Problem Computationally
  11. The Process of Defining the Problem Computationally
  12. Step 1: Defining Communication Restrictions.
  13. Step 2: Defining Cooperation Restrictions.
  14. Step 3: Defining Efficiency Aspects
  15. An Applied Use Case on Artificial Agents
  16. ...and 28 more sections

Figures (5)

  • Figure 1: The task-oriented information flow cycle for shared understanding. One agent, i.e. the Teacher, understands the task that has to be performed, while the other agent i.e. the Student, does not, while the latter is the only one who can perform the task. The agents engage in an iterative loop: 1. Grounded Example: the Teacher attempts to explain the task at hand to the Student providing an example; 2. Action: the Student follows by attempting to perform the task; 3. Understanding Evaluation: the Teacher is informed by the Environment regarding the Student's performance on the task, which reflects its understanding of the task.
  • Figure 2: The shared understanding cycle instantiated for our example. Step (1) Provide example: the Ph.D. student (acting as Teacher) wants to query the agent database (acting as Student) about senior researchers ("cmt:ProgramCommitteeMember"). Examples (URIs for senior and junior researcher) both the Ph.D. student and the database agent are aware of those common objects. Step (2) Forward Query: the agent database needs to interpret the example and apply its Student policy to update its current estimated query representation, and forward the query results from its database to the supervisors (Environment). Step (3) Return Query Performance: evaluation in terms of Precision and Recall, and inform their student how well the agent database understands her query.
  • Figure 3: Example of how two agents can perceive the same grounded example (URIs) differently according to their ontologies. The URIs are grounded: both ontologies refer to the same "entities".
  • Figure 4: Performance evaluation of the proposed agent policies on the two datasets. The legends denote the Teacher and Student applied policy respectively, separated by '-'.
  • Figure :