Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ImmersiveFlow: Stereo-to-7.1.4 spatial audio generation with flow matching

Zining Liang, Runbang Wang, Xuzhou Ye, Qiuqiang Kong

TL;DR

ImmersiveFlow addresses the challenge of generating discrete 7.1.4 spatial audio directly from stereo, overcoming binaural limitations and FOA spatial aliasing. The method employs Conditional Flow Matching to learn a velocity field $v_ heta(oldsymbol{z}_t, t, oldsymbol{z}_{cond})$ with $ rac{d oldsymbol{z}_t}{dt} = v_ heta(oldsymbol{z}_t, t, oldsymbol{z}_{cond})$, trained to minimize $oxed{oxed{ ext{L}_{Flow}}}$. A pretrained VAE encodes stereo and 7.1.4 audio into latent spaces; during inference, the Dormand–Prince ODE solver integrates the latent trajectory to $t=1$, after which the VAE decoder reconstructs the 7.1.4 waveform. Experiments on a privately collected dataset show ImmersiveFlow yields competitive perceptual quality while achieving superior generative metrics (FAD, MAD) compared with upmix baselines, indicating improved spatial fidelity and externalization for high-channel playback.

Abstract

Immersive spatial audio has become increasingly critical for applications ranging from AR/VR to home entertainment and automotive sound systems. However, existing generative methods remain constrained to low-dimensional formats such as binaural audio and First-Order Ambisonics (FOA). Binaural rendering is inherently limited to headphone playback, while FOA suffers from spatial aliasing and insufficient resolution for high-frequency. To overcome these limitations, we introduce ImmersiveFlow, the first end-to-end generative framework that directly synthesizes discrete 7.1.4 format spatial audio from stereo input. ImmersiveFlow leverages Flow Matching to learn trajectories from stereo inputs to multichannel spatial features within a pretrained VAE latent space. At inference, the Flow Matching model predicted latent features are decoded by the VAE and converted into the final 7.1.4 waveform. Comprehensive objective and subjective evaluations demonstrate that our method produces perceptually rich sound fields and enhanced externalization, significantly outperforming traditional upmixing techniques. Code implementations and audio samples are provided at: https://github.com/violet-audio/ImmersiveFlow.

ImmersiveFlow: Stereo-to-7.1.4 spatial audio generation with flow matching

TL;DR

ImmersiveFlow addresses the challenge of generating discrete 7.1.4 spatial audio directly from stereo, overcoming binaural limitations and FOA spatial aliasing. The method employs Conditional Flow Matching to learn a velocity field with , trained to minimize . A pretrained VAE encodes stereo and 7.1.4 audio into latent spaces; during inference, the Dormand–Prince ODE solver integrates the latent trajectory to , after which the VAE decoder reconstructs the 7.1.4 waveform. Experiments on a privately collected dataset show ImmersiveFlow yields competitive perceptual quality while achieving superior generative metrics (FAD, MAD) compared with upmix baselines, indicating improved spatial fidelity and externalization for high-channel playback.

Abstract

Immersive spatial audio has become increasingly critical for applications ranging from AR/VR to home entertainment and automotive sound systems. However, existing generative methods remain constrained to low-dimensional formats such as binaural audio and First-Order Ambisonics (FOA). Binaural rendering is inherently limited to headphone playback, while FOA suffers from spatial aliasing and insufficient resolution for high-frequency. To overcome these limitations, we introduce ImmersiveFlow, the first end-to-end generative framework that directly synthesizes discrete 7.1.4 format spatial audio from stereo input. ImmersiveFlow leverages Flow Matching to learn trajectories from stereo inputs to multichannel spatial features within a pretrained VAE latent space. At inference, the Flow Matching model predicted latent features are decoded by the VAE and converted into the final 7.1.4 waveform. Comprehensive objective and subjective evaluations demonstrate that our method produces perceptually rich sound fields and enhanced externalization, significantly outperforming traditional upmixing techniques. Code implementations and audio samples are provided at: https://github.com/violet-audio/ImmersiveFlow.
Paper Structure (12 sections, 5 equations, 3 figures, 2 tables)

This paper contains 12 sections, 5 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. METHOD
  3. VAE Encoder and Decoder
  4. Flow Matching for Immersive Sound Generation
  5. Solving the Ordinary Differential Equation for Inference
  6. EVALUATION
  7. Dataset
  8. Evaluation Metrics
  9. Experiments Details
  10. Results
  11. Case Studies
  12. CONCLUSION

Figures (3)

  • Figure 1: Overview of the ImmersiveFlow architecture. During training, both stereo and 7.1.4 immersive audio are encoded into latent using a pretrained VAE encoder. The model learns to map stereo latents to immersive sound latents using Flow Matching. During inference, only the stereo is input to predict immersive sound latents, which are then decoded to 7.1.4 audio using the VAE decoder.
  • Figure 2: Illustration of the 7.1.4 loudspeaker configuration with positions, following ITU standard ITU-RBS2051-3. Abbreviations: L/R (Left/Right), C (Center), LFE (Subwoofer), Lss/Rss (Side Surround), Lrs/Rrs (Rear Surround), Ltf/Rtf (Top Front), Ltb/Rtb (Top Back). The positions shown are [azimuth, elevation].
  • Figure 3: Log-mel spectrograms of a 10-second example from the test set. (a) Ground truth 7.1.4 audio; (b) ImmersiveFlow generated audio.