Whispering Water: Materializing Human-AI Dialogue as Interactive Ripples

TL;DR

The paper addresses translating human–AI dialogue into observable physical phenomena by using water as a material medium. It proposes a four-phase workflow—confession, contemplation, response, and release—where sentiment primes the water and semantic content drives a six-agent dialogue whose outputs are decomposed into six vibration components and reconstructed in water. The contributions include a novel speech-to-water translation algorithm that leverages STFT-based decomposition and Bark-scale spacing to map linguistic content to cymatic patterns, a dynamic multi-agent system with emergent agent identities through discourse and adaptive voice personas, and a real-time, privacy-conscious hardware–software integration for embodied interaction. The work demonstrates how sensory-rich, material interfaces can render abstract conversational reasoning tangible, with implications for emotional self-exploration, therapy, and intimate HCI.

Abstract

Across cultures, water has served as a recipient of human confession, a yielding medium that receives vulnerability where rigid surfaces cannot. We present Whispering Water, an interactive installation that materializes human-AI dialogue through cymatic patterns on water. Participants confess secrets to a water surface, triggering a four-phase ritual: confession, contemplation, response, and release. The user's speech sentiment is directly transmitted into the water to prime its state, while semantic content enters a multi-agent system, initiating ripples of conversation where agent identities are situated through discourse and voice profiles are chosen based on what they say. We propose a novel algorithm that decomposes speech into component waves and reconstructs them in water, establishing a translation between speech and the physics of material form. By rendering machine reasoning as emergent physical phenomena, the installation explores possibilities for emotional self-exploration through ambiguous, sensory-rich interfaces.

Figures (4)

• Figure 1: The microphone captures the user's voice, driving both an emotion recognition model and a semantic dialogue model. Emotion is classified and mapped to corresponding frequencies. The semantic content passes through multiple rounds of AI agent dialogue, then translates into subwoofers vibration frequencies sent to the installation via TouchDesigner.
• Figure 2: Cymatic patterns across ritual stages. (1) Confession stage: the water surface remains still as the participant confesses. (2) Contemplation stage: the water surface responds to the emotional sentiment detected in the participant's speech. (3-6) Response stage: (3) individual agent generates localized cymatic patterns corresponding to their spatial placement; (4-5) multiple agents engage in dialogue, their speech interfering and creating complex overlapping patterns as they comment on and respond to each other; (6) the summarizer agent performs reconciliation, decomposing the collective response into six component waves that fill the entire tank as the final unified response.
• Figure 3: The system processes speech through sentiment analysis and multi-agent dialogue, both routed through TouchDesigner to six subwoofers arranged by frequency bands (20–40 Hz center, 50–70 Hz intermediate, 80–100 Hz outer). FDM simulations show how subwoofers superposition creates evolving interference patterns, guiding the tank design where six subwoofers are mechanically coupled beneath the water vessel.
• Figure 4: Machine speech decomposition and reconstruction. (A) Original waveform of approximately 3-second machine-synthesized audio. (B) Comparison between linear and log decomposition methods, showing linear harmonics (grey dashed lines) versus Bark-spaced frequency bands (black lines). (C) IFFT reconstruction using linear harmonic components. (D) Water-based reconstruction simulation using log-spaced (Bark scale) method. The visualization demonstrates that the log + Bark approach covers a broader frequency range aligned with human auditory perception, resulting in richer information content in the reconstructed signal.