Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Direction-Aware Neural Acoustic Fields for Few-Shot Interpolation of Ambisonic Impulse Responses

Christopher Ick, Gordon Wichern, Yoshiki Masuyama, François Germain, Jonathan Le Roux

TL;DR

The paper addresses interpolation of spatial room impulse responses with explicit directional information, enabling direction-aware rendering for Ambisonic audio. It introduces Direction-Aware Neural Field (DANF), which models Ambisonic RIRs and uses a direction-aware intensity vector loss to enforce accurate DoA representation. The authors show that the IV loss improves directional metrics and correlates with target DoA while balancing non-directional room metrics, and demonstrate few-shot adaptation to new rooms using LoRA. The work advances immersive audio by enabling efficient, directionally faithful RIR synthesis and adaptable performance across unseen environments.

Abstract

The characteristics of a sound field are intrinsically linked to the geometric and spatial properties of the environment surrounding a sound source and a listener. The physics of sound propagation is captured in a time-domain signal known as a room impulse response (RIR). Prior work using neural fields (NFs) has allowed learning spatially-continuous representations of RIRs from finite RIR measurements. However, previous NF-based methods have focused on monaural omnidirectional or at most binaural listeners, which does not precisely capture the directional characteristics of a real sound field at a single point. We propose a direction-aware neural field (DANF) that more explicitly incorporates the directional information by Ambisonic-format RIRs. While DANF inherently captures spatial relations between sources and listeners, we further propose a direction-aware loss. In addition, we investigate the ability of DANF to adapt to new rooms in various ways including low-rank adaptation.

Direction-Aware Neural Acoustic Fields for Few-Shot Interpolation of Ambisonic Impulse Responses

TL;DR

The paper addresses interpolation of spatial room impulse responses with explicit directional information, enabling direction-aware rendering for Ambisonic audio. It introduces Direction-Aware Neural Field (DANF), which models Ambisonic RIRs and uses a direction-aware intensity vector loss to enforce accurate DoA representation. The authors show that the IV loss improves directional metrics and correlates with target DoA while balancing non-directional room metrics, and demonstrate few-shot adaptation to new rooms using LoRA. The work advances immersive audio by enabling efficient, directionally faithful RIR synthesis and adaptable performance across unseen environments.

Abstract

The characteristics of a sound field are intrinsically linked to the geometric and spatial properties of the environment surrounding a sound source and a listener. The physics of sound propagation is captured in a time-domain signal known as a room impulse response (RIR). Prior work using neural fields (NFs) has allowed learning spatially-continuous representations of RIRs from finite RIR measurements. However, previous NF-based methods have focused on monaural omnidirectional or at most binaural listeners, which does not precisely capture the directional characteristics of a real sound field at a single point. We propose a direction-aware neural field (DANF) that more explicitly incorporates the directional information by Ambisonic-format RIRs. While DANF inherently captures spatial relations between sources and listeners, we further propose a direction-aware loss. In addition, we investigate the ability of DANF to adapt to new rooms in various ways including low-rank adaptation.
Paper Structure (14 sections, 7 equations, 3 figures, 2 tables)

This paper contains 14 sections, 7 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Ambisonics Room Impulse Responses
  3. Methods
  4. Neural Acoustic Fields
  5. Direction-Aware Neural Field
  6. Losses
  7. Experiments
  8. Dataset
  9. Single-Environment Performance
  10. Adaptation
  11. Results
  12. Spatial Improvements
  13. Fine-tuning
  14. Conclusion

Figures (3)

  • Figure 1: Overview of the proposed DANF framework. The input is the environmental context in the form of $K$ bounce points $c_k \in \mathbb{R}^3$, as well as the source and listener locations $s, l \in \mathbb{R}^3$, all of which are concatenated into spatial feature $\tilde{C}$ (denoted by operator $\oplus$) used to create a spatial-temporal encoding $E \in \mathbb{R}^{3K \times T}$. For the impulse generation module, the spatio-temporal embedding $E$ and the orientation of the listener $\theta \in [0,2\pi)$ are decoded into an Ambisonics impulse response $h \in \mathbb{R}^{4 \times T}$.
  • Figure 2: Single-room performance for different values of $\lambda$ (row) and metrics (columns). Each metric is normalized to $[0,1]$ for ease of visualization.
  • Figure 3: Model performance on single rooms with varying intensity vector loss weight $\lambda$, with the dashed line representing no intensity vector loss, or $\lambda=0$.