Paint by Odor: An Exploration of Odor Visualization through Large Language Model and Generative AI

TL;DR

The paper addresses the challenge of conveying olfactory information beyond spatial constraints by introducing Paint by Odor, a pipeline that leverages large language models and generative AI to transform odor descriptions into richly detailed visual art. It proposes design principles for multi-element representations, language-mediated translation, and hierarchical composition, implemented via four modules and seven prompts to generate visuals with Midjourney. Through a preliminary study comparing human and GPT-4 odor descriptors and a formal study evaluating 540 images across 20 real-world odors, the authors examine how visual elements, descriptive language, and abstraction styles affect olfactory synesthesia and aesthetics. The findings reveal that figurative visuals yield higher odor correspondence while abstract styles invite richer interpretation, with language descriptions playing a key mediating role; automation via prompts shows promise for scalable, on-demand odor visualization, albeit with limitations in cultural generalizability and perceptual nuance. The work lays groundwork for future integrations with real-time sensing, personalized prompts, and culturally aware mappings, enabling new design tools for marketing, art, accessibility, and multisensory experiences.

Abstract

Odor visualization translates odor information and perception into visual outcomes and arouses the corresponding olfactory synesthesia, surpassing the spatial limitation that odors can only be perceived where they are present. Traditional odor visualization has typically relied on unidimensional mappings, such as odor-to-color associations, and has required extensive manual design efforts. However, the advent of generative AI (Gen AI) and large language models (LLMs) presents a new opportunity for automatic odor visualization. Nonetheless, gaps remain in bridging olfactory perception with generative tools to produce odor images. To address these gaps, this paper introduces Paint by Odor, a pipeline that leverages Gen AI and LLMs to transform olfactory perceptions into rich, aesthetically engaging visual representations. Two experiments were conducted, where 30 participants smelled real-world odors and provided descriptive data and 28 participants evaluated 560 generated odor images through seven systematically designed prompts. Our findings explored the capability of LLMs in producing olfactory perception by comparing it with human responses and revealed the underlying mechanisms and effects of language-based descriptions and several abstraction styles on odor visualization. Our work further discussed the possibility of automatic odor visualization without human participation. These explorations and results have bridged the research gap in odor visualization using LLMs and Gen AI, offering valuable design insights and various possibilities for future applications.

Figures (15)

• Figure 1: The system schematic of Paint by Odor, a pipeline that utilizes LLMs and Gen AI to generate odor images.
• Figure 2: Image examples of Mouthwash smell and corresponding prompts. Each image corresponds to one of seven prompt inputs to Midjourney.
• Figure 3: (a) Picture of the preliminary study. Participants were blindfolded while sniffing odors, and subsequently filling out the questionnaire using the mobile phone. (b) GPT's olfactory perception. Researchers asked GPT the same questions in the questionnaire including choosing odor descriptors and scoring emotions.
• Figure 4: Results of questionnaires. (a) Results of odor descriptors picked by participants. The numbers came from the sum of participants who chose a particular descriptor for a particular odor. The deeper colors are, the more participants chose them. (b) Participants' average scores of emotional descriptors. The deep blue color meant negative emotions (score 1 3), while the deep red color meant positive emotions (score 5 7). The light gray area represented neutral emotions (score 3 5).
• Figure 5: Image examples of grape smell. For each prompt, we showed all four variations from one generation. There are obvious differences between each image group. For instance, Prompt 1 PureVis illustrates a figurative style, and Prompt 6 RothkoVis represents a cubic and color-composed abstract style. Participants were asked to give an overall evaluation based on four images of each style.