Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Gen-A: Generalizing Ambisonics Neural Encoding to Unseen Microphone Arrays

Mikko Heikkinen, Archontis Politis, Konstantinos Drossos, Tuomas Virtanen

TL;DR

The paper tackles the problem of generalizing Ambisonics encoding to unseen microphone array geometries by introducing a geometry-conditioned neural encoder that predicts a complex mixing matrix $oldsymbol{E}(t,f)$ from MA geometry $oldsymbol{aOmega}$ and captured signals $oldsymbol{x}$. The method employs a U-Net–like architecture with separate geometry and signal encoders; geometry features modulate the signal pathway to produce $oldsymbol{hat} = oldsymbol{E}(t,f)oldsymbol{x}(t,f)$. Evaluations on simulated dry and reverberant scenes show improved encoding fidelity over a conventional static encoder in dry conditions across frequencies, with reverberation presenting more challenge and reducing gains in some bands. The work highlights the potential and limitations of data-driven Ambisonics encoders for irregular and unseen MA geometries and points to further improvements in reverberant environments and perceptual validation, enabling more flexible spatial audio capture systems.

Abstract

Using deep neural networks (DNNs) for encoding of microphone array (MA) signals to the Ambisonics spatial audio format can surpass certain limitations of established conventional methods, but existing DNN-based methods need to be trained separately for each MA. This paper proposes a DNN-based method for Ambisonics encoding that can generalize to arbitrary MA geometries unseen during training. The method takes as inputs the MA geometry and MA signals and uses a multi-level encoder consisting of separate paths for geometry and signal data, where geometry features inform the signal encoder at each level. The method is validated in simulated anechoic and reverberant conditions with one and two sources. The results indicate improvement over conventional encoding across the whole frequency range for dry scenes, while for reverberant scenes the improvement is frequency-dependent.

Gen-A: Generalizing Ambisonics Neural Encoding to Unseen Microphone Arrays

TL;DR

The paper tackles the problem of generalizing Ambisonics encoding to unseen microphone array geometries by introducing a geometry-conditioned neural encoder that predicts a complex mixing matrix from MA geometry and captured signals . The method employs a U-Net–like architecture with separate geometry and signal encoders; geometry features modulate the signal pathway to produce . Evaluations on simulated dry and reverberant scenes show improved encoding fidelity over a conventional static encoder in dry conditions across frequencies, with reverberation presenting more challenge and reducing gains in some bands. The work highlights the potential and limitations of data-driven Ambisonics encoders for irregular and unseen MA geometries and points to further improvements in reverberant environments and perceptual validation, enabling more flexible spatial audio capture systems.

Abstract

Using deep neural networks (DNNs) for encoding of microphone array (MA) signals to the Ambisonics spatial audio format can surpass certain limitations of established conventional methods, but existing DNN-based methods need to be trained separately for each MA. This paper proposes a DNN-based method for Ambisonics encoding that can generalize to arbitrary MA geometries unseen during training. The method takes as inputs the MA geometry and MA signals and uses a multi-level encoder consisting of separate paths for geometry and signal data, where geometry features inform the signal encoder at each level. The method is validated in simulated anechoic and reverberant conditions with one and two sources. The results indicate improvement over conventional encoding across the whole frequency range for dry scenes, while for reverberant scenes the improvement is frequency-dependent.
Paper Structure (10 sections, 6 equations, 2 figures, 1 table)

This paper contains 10 sections, 6 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Proposed Method
  3. DNN Architecture
  4. Evaluation
  5. Data
  6. Employed Hyper-parameters
  7. Baseline
  8. Metrics
  9. Results
  10. Conclusions

Figures (2)

  • Figure 1: Block diagram of the proposed method, where array geometry ($\mathbb{\Omega}$) and array signals ($\mathbf{x}$) are given as input. $\mathbf{E}$ is the estimated signal- and geometry-dependent complex encoding matrix and $\bigotimes$ is used to denote matrix multiplication along the channel dimension.
  • Figure 2: Mean magnitude spectrum error and coherence metrics for proposed and baseline methods in anechoic and reverberant conditions.