Neural Speech Coding for Real-time Communications using Constant Bitrate Scalar Quantization

TL;DR

A new causal network architecture that is based on SQ and a Short-Time Fourier Transform (STFT) representation is proposed that performs particularly well in the very low complexity and low bitrate regime.

Abstract

Neural audio coding has emerged as a vivid research direction by promising good audio quality at very low bitrates unachievable by classical coding techniques. Here, end-to-end trainable autoencoder-like models represent the state of the art, where a discrete representation in the bottleneck of the autoencoder is learned. This allows for efficient transmission of the input audio signal. The learned discrete representation of neural codecs is typically generated by applying a quantizer to the output of the neural encoder. In almost all state-of-the-art neural audio coding approaches, this quantizer is realized as a Vector Quantizer (VQ) and a lot of effort has been spent to alleviate drawbacks of this quantization technique when used together with a neural audio coder. In this paper, we propose and analyze simple alternatives to VQ, which are based on projected Scalar Quantization (SQ). These quantization techniques do not need any additional losses, scheduling parameters or codebook storage thereby simplifying the training of neural audio codecs. For real-time speech communication applications, these neural codecs are required to operate at low complexity, low latency and at low bitrates. We address those challenges by proposing a new causal network architecture that is based on SQ and a Short-Time Fourier Transform (STFT) representation. The proposed method performs particularly well in the very low complexity and low bitrate regime.

Figures (12)

• Figure 1: Generic overview of a neural audio codec: The input signal frame $\mathbf{x}_n$ is processed by a neural encoder $\mathtt{enc}$ yielding a latent representation $\mathbf{y}_n$. This latent representation is mapped to a discrete representation $\mathbf{q}_n$ which is used for transmission. The decoder part receives the discrete representation and dequantizes it by $\mathtt{Q}_{\text{dec}}$. The result is fed to a neural decoder $\mathtt{dec}$ which reconstructs the input signal frame.
• Figure 2: Illustration of an RVQ.
• Figure 3: Quantization of a toy data set (shown in white) with the proposed SQ-based method using three bits. The decision regions corresponding to each codebook vector is shown with a different color.
• Figure 4: Proposed neural speech codec architecture: Variables in brackets denote the output channels and number of output samples/frames.
• Figure 5: Convolutional network blocks: (a) Design of ConvBlocks ($\mathtt{BN}$ is not used for the EncBlocks) used in the (b) EncBlocks and (c) DecBlocks.