Enhancing LLM-Based Human-Robot Interaction with Nuances for Diversity Awareness

TL;DR

A system for diversity-aware autonomous conversation leveraging the capabilities of large language models (LLMs) that adapts to diverse populations and individuals, considering factors like background, personality, age, gender, and culture is presented.

Abstract

This paper presents a system for diversity-aware autonomous conversation leveraging the capabilities of large language models (LLMs). The system adapts to diverse populations and individuals, considering factors like background, personality, age, gender, and culture. The conversation flow is guided by the structure of the system's pre-established knowledge base, while LLMs are tasked with various functions, including generating diversity-aware sentences. Achieving diversity-awareness involves providing carefully crafted prompts to the models, incorporating comprehensive information about users, conversation history, contextual details, and specific guidelines. To assess the system's performance, we conducted both controlled and real-world experiments, measuring a wide range of performance indicators.

Figures (6)

• Figure 1: CAIR system architecture integrating LLMs and dense captioning. Modified or added blocks are colored in blue, while modified or added messages are highlighted in bold.
• Figure 2: Sequence diagram depicting the requests performed from the Dialogue Manager to the LLM.
• Figure 3: System field of the reply request. The content in brackets is replaced by the system with the actual values.
• Figure 4: Evolution of the usage of tone nuance values across the first 100 turns using humorous sentences.
• Figure 5: Time interval between the conclusion of the client's utterance of the filler sentence and the commencement of the reply sentence. The red line represents the average, while the dotted red line indicates $3\sigma$.