PromptReverb: Multimodal Room Impulse Response Generation Through Latent Rectified Flow Matching

TL;DR

PromptReverb tackles the challenge of generating perceptually accurate, full-band room impulse responses from natural language while mitigating data scarcity. It combines a $\beta$-VAE-based upsampling to 48 kHz with a latent Rectified Flow Matching diffusion transformer conditioned on text produced by a caption-then-rewrite pipeline. The method achieves competitive and superior RT60 accuracy (mean errors as low as $8.8\%$ for long prompts and $4.8\%$ for short prompts) and improves subjective reverb quality compared with baselines. This enables flexible, high-quality RIR synthesis for VR, architectural acoustics, and audio production without panoramic imagery or manual acoustic parameter specification.

Abstract

Room impulse response (RIR) generation remains a critical challenge for creating immersive virtual acoustic environments. Current methods suffer from two fundamental limitations: the scarcity of full-band RIR datasets and the inability of existing models to generate acoustically accurate responses from diverse input modalities. We present PromptReverb, a two-stage generative framework that addresses these challenges. Our approach combines a variational autoencoder that upsamples band-limited RIRs to full-band quality (48 kHz), and a conditional diffusion transformer model based on rectified flow matching that generates RIRs from descriptions in natural language. Empirical evaluation demonstrates that PromptReverb produces RIRs with superior perceptual quality and acoustic accuracy compared to existing methods, achieving 8.8% mean RT60 error compared to -37% for widely used baselines and yielding more realistic room-acoustic parameters. Our method enables practical applications in virtual reality, architectural acoustics, and audio production where flexible, high-quality RIR synthesis is essential.

Figures (2)

• Figure 1: PromptReverb system. (a) A VAE pretraining stage that learns an encoder (Enc.) to produce compact latent representations and a decoder (Dec.) to upsample all impulse responses to 48 kHz; (b) A caption-then-rewrite pipeline that employs vision-language models (VLM) to generate descriptions of visual scenes, followed by large language models (LLM) that transform these descriptions into diverse textual prompts; (c) A latent rectified flow matching (RFM) model that generates reverb characteristics in the latent space, conditioned on the text descriptions produced in stage (b).
• Figure 2: Subjective evaluation results showing mean ratings for reverb quality and text-audio matching. PromptReverb outperforms ground truth on both measures and Image2Reverb on quality.