Data Augmentation Using Neural Acoustic Fields With Retrieval-Augmented Pre-training
Christopher Ick, Gordon Wichern, Yoshiki Masuyama, François G. Germain, Jonathan Le Roux
TL;DR
The paper tackles estimating room impulse responses (RIRs) at unseen source–receiver locations and improving speaker distance estimation under GenDARA Tasks 1 and 2. It introduces retrieval-augmented pre-training of a neural acoustic field conditioned on room geometry by leveraging the large external GWA RIR dataset and 3D-FRONT room meshes, followed by room-specific fine-tuning with low-rank adaptation (LoRA) where updates follow $ar{W} = W + BA^T$ of rank $r$. A neural field based on a simplified INRAS-like architecture accepts $(s,r)$ coordinates and $K$ bounce points sampled from the room surface via Poisson disk sampling, with sinusoidal encodings to predict the RIR $h(s,r)$. Experiments demonstrate that retrieval-based pre-training on the GWA dataset with LoRA improves EDF and DRR metrics over baselines, and the generated RIRs enable effective training of the downstream SDE-based speaker distance estimator, highlighting the practical impact for data augmentation in acoustic tasks.
Abstract
This report details MERL's system for room impulse response (RIR) estimation submitted to the Generative Data Augmentation Workshop at ICASSP 2025 for Augmenting RIR Data (Task 1) and Improving Speaker Distance Estimation (Task 2). We first pre-train a neural acoustic field conditioned by room geometry on an external large-scale dataset in which pairs of RIRs and the geometries are provided. The neural acoustic field is then adapted to each target room by using the enrollment data, where we leverage either the provided room geometries or geometries retrieved from the external dataset, depending on availability. Lastly, we predict the RIRs for each pair of source and receiver locations specified by Task 1, and use these RIRs to train the speaker distance estimation model in Task 2.