Overview of Dialogue Robot Competition 2023

TL;DR

This work surveys DRC2023, a two-stage, on-site competition evaluating humanoid dialogue robots in a travel-itinerary planning task. It emphasizes the shift toward real-time, multimodal, LLM-enabled dialogue within actual travel-agency environments and documents the preliminary round results, including participant diversity, evaluation metrics, and finalist performance. The study highlights how integrated LLM strategies, routing tools, and nonverbal interaction contribute to customer-perceived impression and feasibility of travel plans, illustrating practical progress toward hospitality-oriented dialogue robots. The final round and subsequent analysis are expected to further delineate effective architectural patterns for real-world, on-site dialogue systems.

Abstract

We have held dialogue robot competitions in 2020 and 2022 to compare the performances of interactive robots using an android that closely resembles a human. In 2023, the third competition DRC2023 was held. The task of DRC2023 was designed to be more challenging than the previous travel agent dialogue tasks. Since anyone can now develop a dialogue system using LLMs, the participating teams are required to develop a system that effectively uses information about the situation on the spot (real-time information), which is not handled by ChatGPT and other systems. DRC2023 has two rounds, a preliminary round and the final round as well as the previous competitions. The preliminary round has held on Oct.27 -- Nov.20, 2023 at real travel agency stores. The final round will be held on December 23, 2023. This paper provides an overview of the task settings and evaluation method of DRC2023 and the preliminary round results.

Figures (8)

• Figure 1: An example of display of sightseeing spots on the monitor
• Figure 2: Android I used in the competition
• Figure 3: Overview of the middleware distributed to participating teams
• Figure 4: Preliminary round at JTB stores
• Figure 5: Overview of baseline system