Role-Play Zero-Shot Prompting with Large Language Models for Open-Domain Human-Machine Conversation
Ahmed Njifenjou, Virgile Sucal, Bassam Jabaian, Fabrice Lefèvre
TL;DR
The paper tackles the challenge of building open-domain conversational agents with large language models without costly finetuning. It introduces role-play zero-shot prompting for instruction-following models to imbue agents with persona, empathy, and conversational engagement, evaluated in French on PersonaChat and INT tasks. The authors present a general prompt structure and a per-turn prompt-builder that integrates system directives, context, and history, demonstrating results that rival or exceed finetuned baselines. They validate the approach across self-chats and human-bot interactions, discuss limitations such as hallucinations, and outline future work including automated prompt generation and reinforcement learning to adapt prompts during dialogue, with evidence of good multilingual transfer for Vicuna-based systems.
Abstract
Recently, various methods have been proposed to create open-domain conversational agents with Large Language Models (LLMs). These models are able to answer user queries, but in a one-way Q&A format rather than a true conversation. Fine-tuning on particular datasets is the usual way to modify their style to increase conversational ability, but this is expensive and usually only available in a few languages. In this study, we explore role-play zero-shot prompting as an efficient and cost-effective solution for open-domain conversation, using capable multilingual LLMs (Beeching et al., 2023) trained to obey instructions. We design a prompting system that, when combined with an instruction-following model - here Vicuna (Chiang et al., 2023) - produces conversational agents that match and even surpass fine-tuned models in human evaluation in French in two different tasks.