Emotion-Aware Conversational Recommender Systems: a Case Study
Maria Stella Albarelli
TL;DR
This study introduces Gala, an emotion-aware conversational recommender for Galeries Lafayette aimed at recreating in-store empathic guidance in online fashion shopping. Gala leverages voice emotion recognition (via Emoty), NLP-powered dialog with GPT-4o, and a content-based product search integrated with the Galeries Lafayette API, enabling context-sensitive, multi-turn interactions. The design process combined online and in-store interviews to identify user needs, followed by two user studies (usability and empathy evaluation) and iteration on a high-fidelity prototype; results indicate that emotion-aware interaction can enhance speed, engagement, and perceived similarity to in-store assistance, though challenges remain in emotion detection reliability and latency. The work provides a practical blueprint for deploying emotion-aware CRS in fashion retail and outlines concrete improvements, including stronger emotion-recognition robustness, latency reduction, multimodal prompting, and potential in-store Gala avatars for cross-channel experiences.
Abstract
In recent years, online shopping has grown rapidly, especially during the COVID-19 period. However, it still lacks elements typical of physical stores, such as empathic support and personalised advice from a sales assistant. This study explores how an emotion-aware Conversational Agent (CA) can improve the online shopping experience by responding to user emotions in a more natural and human way. The project focuses on Gala, a virtual assistant developed for the Galeries Lafayette website, capable of recognising emotional states from voice messages and adapting its responses accordingly. User needs were first analysed through semi-structured interviews, which informed the design of Gala's UX and functionalities. Gala was implemented using the OpenAI API and the Galeries Lafayette API, adopting a Content-Based recommendation approach. Through Natural Language Processing, it interprets user requests and retrieves products aligned with specific attributes such as name, price, and brand, enabling fluid dialogue and tailored suggestions. Two user studies were conducted: a usability test and a comparative evaluation between a standard CA and Gala's emotion-aware version. The results highlight the potential of emotion-aware CAs to make online shopping faster, more engaging, and closer to an in-store guided experience.