Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Steerer: Novel Steering Vector Synthesis with a Causal Neural Field over Frequency and Source Positions

Diego Di Carlo, Aditya Arie Nugraha, Mathieu Fontaine, Mathieu Fontaine, Kazuyoshi Yoshii

TL;DR

This work addresses the problem of accurately interpolating measured anechoic steering vectors with a deep learning framework called the neural field, and introduces the neural steerer, a continuous complex-valued function that takes both frequency and direction as input and produces the corresponding steering vector.

Abstract

We address the problem of accurately interpolating measured anechoic steering vectors with a deep learning framework called the neural field. This task plays a pivotal role in reducing the resource-intensive measurements required for precise sound source separation and localization, essential as the front-end of speech recognition. Classical approaches to interpolation rely on linear weighting of nearby measurements in space on a fixed, discrete set of frequencies. Drawing inspiration from the success of neural fields for novel view synthesis in computer vision, we introduce the neural steerer, a continuous complex-valued function that takes both frequency and direction as input and produces the corresponding steering vector. Importantly, it incorporates inter-channel phase difference information and a regularization term enforcing filter causality, essential for accurate steering vector modeling. Our experiments, conducted using a dataset of real measured steering vectors, demonstrate the effectiveness of our resolution-free model in interpolating such measurements.

Neural Steerer: Novel Steering Vector Synthesis with a Causal Neural Field over Frequency and Source Positions

TL;DR

This work addresses the problem of accurately interpolating measured anechoic steering vectors with a deep learning framework called the neural field, and introduces the neural steerer, a continuous complex-valued function that takes both frequency and direction as input and produces the corresponding steering vector.

Abstract

We address the problem of accurately interpolating measured anechoic steering vectors with a deep learning framework called the neural field. This task plays a pivotal role in reducing the resource-intensive measurements required for precise sound source separation and localization, essential as the front-end of speech recognition. Classical approaches to interpolation rely on linear weighting of nearby measurements in space on a fixed, discrete set of frequencies. Drawing inspiration from the success of neural fields for novel view synthesis in computer vision, we introduce the neural steerer, a continuous complex-valued function that takes both frequency and direction as input and produces the corresponding steering vector. Importantly, it incorporates inter-channel phase difference information and a regularization term enforcing filter causality, essential for accurate steering vector modeling. Our experiments, conducted using a dataset of real measured steering vectors, demonstrate the effectiveness of our resolution-free model in interpolating such measurements.
Paper Structure (8 sections, 6 equations, 4 figures, 1 table)

This paper contains 8 sections, 6 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Method
  4. Neural Steerer
  5. Network Architecture
  6. Training Loss and Off-Grid Regularization
  7. Evaluation
  8. Conclusion

Figures (4)

  • Figure 1: Schematic representation of the proposed Neural Steerer for measured anechoic steering vectors interpolation.
  • Figure 2: Illustration of the proposed Neural Steerer. Both $\odot$ and $\cdot$ denote element-wise multiplication.
  • Figure 3: RMSE (left) and cosine distance (right) of time-domain steering vectors for locations in the test set (top) and the whole sphere reconstruction on random data (bottom). Shaded regions show the confidence interval for both continuous and discrete frequency models.
  • Figure 4: Average log-spectral distortion at different resolutions in the whole (left) and a selected (right) frequency range. The model was trained on $F=257$ frequency bins.