Compression of Higher Order Ambisonics with Multichannel RVQGAN

TL;DR

A multichannel extension to the RVQGAN neural coding method is proposed, and realized for data-driven compression of third-order Ambisonics audio and a loss function for accounting for spatial perception in immersive reproduction, and transfer learning from single-channel models.

Abstract

A multichannel extension to the RVQGAN neural coding method is proposed, and realized for data-driven compression of third-order Ambisonics audio. The input- and output layers of the generator and discriminator models are modified to accept multiple (16) channels without increasing the model bitrate. We also propose a loss function for accounting for spatial perception in immersive reproduction, and transfer learning from single-channel models. Listening test results with 7.1.4 immersive playback show that the proposed extension is suitable for coding scene-based, 16-channel Ambisonics content with good quality at 16 kbps when trained and tested on the EigenScape database. The model has potential applications for learning other types of content and multichannel formats.

Figures (2)

• Figure 1: Example comparison between proposed transfer learning utilizing pre-trained single-channel model weights, and vanilla random initialization. Plot shows the development of the reconstruction multi-scale mel validation loss descript.
• Figure 2: MUSHRA score mean and 95% confidence interval over 8 tracks and 8 listeners, with 7.1.4 immersive loudspeaker listening.