Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving Opus Low Bit Rate Quality with Neural Speech Synthesis

Jan Skoglund, Jean-Marc Valin

TL;DR

The paper tackles poor quality of Opus at very low bitrates by introducing backward-compatible neural synthesis using decoded Opus parameters. It compares WaveNet (upper-bound quality) and LPCNet (real-time capable) as refinements applied to Opus features, demonstrating substantial quality gains at 6 kb/s. WaveNet can match Opus at 9 kb/s, while LPCNet provides substantial improvement between 6 and 9 kb/s with real-time feasibility on mobile devices. This approach shows that neural vocoding can extend the utility of existing codecs without changing their bitstreams, with potential application to other waveform coders.

Abstract

The voice mode of the Opus audio coder can compress wideband speech at bit rates ranging from 6 kb/s to 40 kb/s. However, Opus is at its core a waveform matching coder, and as the rate drops below 10 kb/s, quality degrades quickly. As the rate reduces even further, parametric coders tend to perform better than waveform coders. In this paper we propose a backward-compatible way of improving low bit rate Opus quality by re-synthesizing speech from the decoded parameters. We compare two different neural generative models, WaveNet and LPCNet. WaveNet is a powerful, high-complexity, and high-latency architecture that is not feasible for a practical system, yet provides a best known achievable quality with generative models. LPCNet is a low-complexity, low-latency RNN-based generative model, and practically implementable on mobile phones. We apply these systems with parameters from Opus coded at 6 kb/s as conditioning features for the generative models. A listening test shows that for the same 6 kb/s Opus bit stream, synthesized speech using LPCNet clearly outperforms the output of the standard Opus decoder. This opens up ways to improve the decoding quality of existing speech and audio waveform coders without breaking compatibility.

Improving Opus Low Bit Rate Quality with Neural Speech Synthesis

TL;DR

The paper tackles poor quality of Opus at very low bitrates by introducing backward-compatible neural synthesis using decoded Opus parameters. It compares WaveNet (upper-bound quality) and LPCNet (real-time capable) as refinements applied to Opus features, demonstrating substantial quality gains at 6 kb/s. WaveNet can match Opus at 9 kb/s, while LPCNet provides substantial improvement between 6 and 9 kb/s with real-time feasibility on mobile devices. This approach shows that neural vocoding can extend the utility of existing codecs without changing their bitstreams, with potential application to other waveform coders.

Abstract

The voice mode of the Opus audio coder can compress wideband speech at bit rates ranging from 6 kb/s to 40 kb/s. However, Opus is at its core a waveform matching coder, and as the rate drops below 10 kb/s, quality degrades quickly. As the rate reduces even further, parametric coders tend to perform better than waveform coders. In this paper we propose a backward-compatible way of improving low bit rate Opus quality by re-synthesizing speech from the decoded parameters. We compare two different neural generative models, WaveNet and LPCNet. WaveNet is a powerful, high-complexity, and high-latency architecture that is not feasible for a practical system, yet provides a best known achievable quality with generative models. LPCNet is a low-complexity, low-latency RNN-based generative model, and practically implementable on mobile phones. We apply these systems with parameters from Opus coded at 6 kb/s as conditioning features for the generative models. A listening test shows that for the same 6 kb/s Opus bit stream, synthesized speech using LPCNet clearly outperforms the output of the standard Opus decoder. This opens up ways to improve the decoding quality of existing speech and audio waveform coders without breaking compatibility.

Paper Structure

This paper contains 10 sections, 3 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Opus speech compression
  3. Autoregressive Generative Networks for Speech Synthesis
  4. WaveNet
  5. LPCNet
  6. Conditioning Features and Training
  7. Evaluation
  8. Experimental Setup
  9. Results
  10. Conclusion

Figures (2)

  • Figure 1: Overview of the LPCNet model. The frame rate network (yellow) operates on frame-wise representing features and its output is held constant through each frame for the sample rate network (blue). The compute prediction block applies linear prediction to predict the sample at time $t$ from the previous samples. Conversions between $\mu$-law and linear are omitted for clarity. The de-emphasis filter is applied to the output $s_{t}$.
  • Figure 2: Listening test results for sets 1 and 2. The error bars indicate a 95% confidence interval.