Sniff AI: Is My 'Spicy' Your 'Spicy'? Exploring LLM's Perceptual Alignment with Human Smell Experiences

TL;DR

A user study to investigate how well AI can interpret human descriptions of scents, with results indicated limited perceptual alignment, with biases towards certain scents and continued failing to identify others.

Abstract

Aligning AI with human intent is important, yet perceptual alignment-how AI interprets what we see, hear, or smell-remains underexplored. This work focuses on olfaction, human smell experiences. We conducted a user study with 40 participants to investigate how well AI can interpret human descriptions of scents. Participants performed "sniff and describe" interactive tasks, with our designed AI system attempting to guess what scent the participants were experiencing based on their descriptions. These tasks evaluated the Large Language Model's (LLMs) contextual understanding and representation of scent relationships within its internal states - high-dimensional embedding space. Both quantitative and qualitative methods were used to evaluate the AI system's performance. Results indicated limited perceptual alignment, with biases towards certain scents, like lemon and peppermint, and continued failing to identify others, like rosemary. We discuss these findings in light of human-AI alignment advancements, highlighting the limitations and opportunities for enhancing HCI systems with multisensory experience integration.

Figures (12)

• Figure 1: SniffAI: an overview of user study and examples from the user study.
• Figure 2: An overview of Sniff AI System components and workflow: starting with a human describing a scent. The description is processed by an ASR system, mapped to an AI embedding space, and compared with pre-built scent embeddings. The AI guessing mechanism predicts the scent and then delivers the guessed scent by a scent delivery device.
• Figure 3: An overview of the "AI sniff" method and the "Guess what scent" user study workflow.
• Figure 4: Study Setups (a) A participant controls the AI guessing system, and sniffs the scent delivered from the device inside a noise isolation box. (b) A close view of the scent delivery device without the noise isolation box (left) and the user interface (right).
• Figure 5: User study procedure in four stages: 1. Prescreening & Demographics, 2. Task 1: Participants sniff scents and describe them, followed by AI predictions, 3. Task 2: Participants compare scents, describe differences, and AI makes further guesses, 4. Concluding Interview.