CloChat: Understanding How People Customize, Interact, and Experience Personas in Large Language Models

TL;DR

This work tackles the rigidity of generic LLM agents by introducing CloChat, a form-based interface that lets users design and interact with personalized agent personas. Through a within-subjects study against ChatGPT (GPT-4) with 30 Korean-speaking participants, CloChat demonstrated higher user satisfaction, richer and more varied dialogues, and stronger emotional connections to customized agents. The findings indicate that persona customization can sustain engagement over time and yield more diverse conversations, with visual representations further enhancing empathy. The paper contributes a practical system for persona design, empirical evidence of benefits and tradeoffs, and design and ethical guidelines to inform future LLM-based conversational interfaces.

Abstract

Large language models (LLMs) have facilitated significant strides in generating conversational agents, enabling seamless, contextually relevant dialogues across diverse topics. However, the existing LLM-driven conversational agents have fixed personalities and functionalities, limiting their adaptability to individual user needs. Creating personalized agent personas with distinct expertise or traits can address this issue. Nonetheless, we lack knowledge of how people customize and interact with agent personas. In this research, we investigated how users customize agent personas and their impact on interaction quality, diversity, and dynamics. To this end, we developed CloChat, an interface supporting easy and accurate customization of agent personas in LLMs. We conducted a study comparing how participants interact with CloChat and ChatGPT. The results indicate that participants formed emotional bonds with the customized agents, engaged in more dynamic dialogues, and showed interest in sustaining interactions. These findings contribute to design implications for future systems with conversational agents using LLMs.

Figures (8)

• Figure 1: CloChat Design Lab Interface Features. The Design Lab interface comprises multiple pages, each linked to one of six categories from our literature review. Users input information into text fields for Demographic Cues (a) and Knowledge and Interest Cues (c). The Verbal Style Cues (b) page offers various language styles, selectable via checkboxes. Emoji options (d) are added through toggle switches. For the Appearance category (f), users describe the visual representation in text, detailed in \ref{['fig:app_architecture']}.
• Figure 2: Technical architecture of CloChat (\ref{['sec:architecture']}). (Step 1) Given the non-visual traits from the CloChat design lab, we first convert them to a JSON specification (purple-filled box). (Step 2) We use GPT-4 to translate the JSON specification into a system message describing a persona (text with an orange background). (Step 3) We inject the system message into GPT-4, making it answer the user’s message from the agent persona's perspective (text with a light-green background).
• Figure 3: Appearance feature of CloChat’s design lab and its technical architecture. When users set the characteristics of the agent persona, they can also create a profile image for that agent. CloChat generates images based on the user's choices, and users can select the image most suitable for the persona they have set up. Additionally, users can further customize the agent's profile image by directly entering text. (Step 1) Once the image prompt written in Korean (text with a light-green background) is received from the design lab, CloChat first translate the prompt into English (text with an orange background) using GPT-4. (Step 2) The image prompt is injected into DALL-E2, which generates four candidate images. The generated images are then presented to the users via the design lab, where they can choose one as the final visual representation (red-bordered image).
• Figure 4: Procedure of our experiment. After the participants a) signed the consent form and (b) participated in a preliminary interview, they interacted with conversational agents using (c) ChatGPT and (d) CloChat. Half of the participants interacted with ChatGPT first, as shown in the figure, while the other half interacted with CloChat first and then with ChatGPT (not shown). The study ended with a (e) post hoc interview.
• Figure 5: Customization process of the agent's persona in CloChat's design lab. Participants customized agent personas in the CloChat Design Lab to suit each scenario. They adjusted options ranging from (a) Demographic Cues to (d) Visual Appearance. Additionally, a preview feature (b) allowed them to preview the persona's responses. Once customization was complete, participants proceeded to the CloChat Room (e) for conversations with their personalized agent.