Before Smelling the Video: A Two-Stage Pipeline for Interpretable Video-to-Scent Plans

TL;DR

The paper addresses the challenge of integrating smell into dynamic video by introducing a two-stage video-to-scent planning pipeline that first extracts visual semantics with a vision–language model and then infers structured olfactory plans via a large language model constrained by a fixed odor schema. Through two online surveys, the authors show that system-generated scent plans are perceived as temporally coherent and plausible, with a strong preference for plans that emphasize a dominant olfactory source and align changes with observable actions. The work demonstrates semantic planning as a foundational step toward scalable olfactory media systems and provides insights for designing future scent-delivery interfaces and user controls. Its findings have practical implications for enhancing immersion in olfactory-enhanced media without requiring immediate physical scent deployment.

Abstract

Olfactory cues can enhance immersion in interactive media, yet smell remains rare because it is difficult to author and synchronize with dynamic video. Prior olfactory interfaces rely on designer triggers and fixed event-to-odor mappings that do not scale to unconstrained content. This work examines whether semantic planning for smell is intelligible to people before physical scent delivery. We present a video-to-scent planning pipeline that separates visual semantic extraction using a vision-language model from semantic-to-olfactory inference using a large language model. Two survey studies compare system-generated scent plans with over-inclusive and naive baselines. Results show consistent preference for plans that prioritize perceptually salient cues and align scent changes with visible actions, supporting semantic planning as a foundation for future olfactory media systems.