Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OnomaCompass: A Texture Exploration Interface that Shuttles between Words and Images

Miki Okamura, Shuhey Koyama, Li Jingjing, Yoichi Ochiai

TL;DR

OnomaCompass introduces a bidirectional, dual-latent-space interface that shuttles between texture images and sound-symbolic onomatopoeia to support early-stage materiality ideation. By linking perceptual textures with invented mimetics and enabling cross-modal highlighting and emergent video interpolation, it reduces the verbalization burden associated with prompt-based AI generation and fosters serendipitous, divergent exploration. A within-subjects study (N=11) shows that OnomaCompass lowers workload and enhances hedonic experience relative to a baseline prompt-based workflow, though usability of 3D navigation lags behind traditional interfaces. Overall, the system positions itself as a complementary tool that helps novices construct a mental map of materiality, enabling explorations of the in-between space between words and images, with future work targeting improved navigation, clearer mapping semantics, and hybrid divergent-convergent workflows.

Abstract

Humans can finely perceive material textures, yet articulating such somatic impressions in words is a cognitive bottleneck in design ideation. We present OnomaCompass, a web-based exploration system that links sound-symbolic onomatopoeia and visual texture representations to support early-stage material discovery. Instead of requiring users to craft precise prompts for generative AI, OnomaCompass provides two coordinated latent-space maps--one for texture images and one for onomatopoeic term--built from an authored dataset of invented onomatopoeia and corresponding textures generated via Stable Diffusion. Users can navigate both spaces, trigger cross-modal highlighting, curate findings in a gallery, and preview textures applied to objects via an image-editing model. The system also supports video interpolation between selected textures and re-embedding of extracted frames to form an emergent exploration loop. We conducted a within-subjects study with 11 participants comparing OnomaCompass to a prompt-based image-generation workflow using Gemini 2.5 Flash Image ("Nano Banana"). OnomaCompass significantly reduced workload (NASA-TLX overall, mental demand, effort, and frustration; p < .05) and increased hedonic user experience (UEQ), while usability (SUS) favored the baseline. Qualitative findings indicate that OnomaCompass helps users externalize vague sensory expectations and promotes serendipitous discovery, but also reveals interaction challenges in spatial navigation. Overall, leveraging sound symbolism as a lightweight cue offers a complementary approach to Kansei-driven material ideation beyond prompt-centric generation.

OnomaCompass: A Texture Exploration Interface that Shuttles between Words and Images

TL;DR

OnomaCompass introduces a bidirectional, dual-latent-space interface that shuttles between texture images and sound-symbolic onomatopoeia to support early-stage materiality ideation. By linking perceptual textures with invented mimetics and enabling cross-modal highlighting and emergent video interpolation, it reduces the verbalization burden associated with prompt-based AI generation and fosters serendipitous, divergent exploration. A within-subjects study (N=11) shows that OnomaCompass lowers workload and enhances hedonic experience relative to a baseline prompt-based workflow, though usability of 3D navigation lags behind traditional interfaces. Overall, the system positions itself as a complementary tool that helps novices construct a mental map of materiality, enabling explorations of the in-between space between words and images, with future work targeting improved navigation, clearer mapping semantics, and hybrid divergent-convergent workflows.

Abstract

Humans can finely perceive material textures, yet articulating such somatic impressions in words is a cognitive bottleneck in design ideation. We present OnomaCompass, a web-based exploration system that links sound-symbolic onomatopoeia and visual texture representations to support early-stage material discovery. Instead of requiring users to craft precise prompts for generative AI, OnomaCompass provides two coordinated latent-space maps--one for texture images and one for onomatopoeic term--built from an authored dataset of invented onomatopoeia and corresponding textures generated via Stable Diffusion. Users can navigate both spaces, trigger cross-modal highlighting, curate findings in a gallery, and preview textures applied to objects via an image-editing model. The system also supports video interpolation between selected textures and re-embedding of extracted frames to form an emergent exploration loop. We conducted a within-subjects study with 11 participants comparing OnomaCompass to a prompt-based image-generation workflow using Gemini 2.5 Flash Image ("Nano Banana"). OnomaCompass significantly reduced workload (NASA-TLX overall, mental demand, effort, and frustration; p < .05) and increased hedonic user experience (UEQ), while usability (SUS) favored the baseline. Qualitative findings indicate that OnomaCompass helps users externalize vague sensory expectations and promotes serendipitous discovery, but also reveals interaction challenges in spatial navigation. Overall, leveraging sound symbolism as a lightweight cue offers a complementary approach to Kansei-driven material ideation beyond prompt-centric generation.
Paper Structure (64 sections, 4 figures)

This paper contains 64 sections, 4 figures.

Table of Contents

  1. Introduction
  2. A new frontier of creativity: Designing “materiality”
  3. The “verbalization barrier” in generative AI
  4. A “shuttling” between words and images: A dialogue that bridges two worlds
  5. Onomatopoeia: A bridge to the sensorial world and an AI’s vision
  6. Proposal and contributions
  7. Related Work
  8. Discreteness and continuity in creative dialogue
  9. Challenges of text prompts: Semantic gap and discrete trial-and-error
  10. Efficacy of continuous interfaces
  11. Navigating the sensorial world: Exploration of latent spaces
  12. Foundations of Unified Representation
  13. Visualization and Manipulation Tools
  14. Ideation-Focused Interfaces
  15. A bridge to sensation via onomatopoeia
  16. ...and 49 more sections

Figures (4)

  • Figure 1: OnomaCompass system overview: dual visualization of image and language latent spaces with cross-modal highlighting, a gallery for curation and texture application to objects, and a dynamic video interpolation and replotting pipeline that feeds newly generated frames back into both spaces.
  • Figure 2: The user interface of the proposed system. (a) Texture Space: A scatter plot of texture images arranged based on visual features. (b) Onomatopoeia Space: A scatter plot of onomatopoeic terms arranged by text-embedding similarity of the LLM-generated prompt descriptions (written to reflect sound-symbolic impressions; e.g., "Honyonyon"). (c) Gallery and Preview: An area for managing selected textures and simulating their application onto a target object (e.g., a vase).
  • Figure 3: Comparative evaluation results using standard metrics. Scores for (a) UEQ (User Experience Questionnaire), (b) NASA-TLX (subjective workload), and (c) SUS (System Usability Scale). In the box plots, the center line represents the median, the dot represents the mean, the box indicates the interquartile range (IQR), and the whiskers represent the range within 1.5 × IQR. (OC: Proposed System [OnomaCompass], NB: Conventional Method [Nano Banana], * p < .05, ** p < .01)
  • Figure 4: Evaluation results from the original questionnaire (Q1–Q12). Response distributions for 12 items (7-point Likert scale) regarding creativity support and exploration experience. OC (Proposed System) received significantly higher ratings, particularly in diverse exploration (Q3), avoidance of getting stuck (Q6), creative process (Q8), and unexpected discoveries (Q10). (* p < .05, ** p < .01)