Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Differentiable Time-Varying Linear Prediction in the Context of End-to-End Analysis-by-Synthesis

Chin-Yun Yu, György Fazekas

TL;DR

It is shown that the improved GOLF learns LP coefficients and reconstructs the voice better than its frame-wise counterparts, and synthesised outputs from GOLF scored higher in quality ratings than the state-of-the-art differentiable WORLD vocoder.

Abstract

Training the linear prediction (LP) operator end-to-end for audio synthesis in modern deep learning frameworks is slow due to its recursive formulation. In addition, frame-wise approximation as an acceleration method cannot generalise well to test time conditions where the LP is computed sample-wise. Efficient differentiable sample-wise LP for end-to-end training is the key to removing this barrier. We generalise the efficient time-invariant LP implementation from the GOLF vocoder to time-varying cases. Combining this with the classic source-filter model, we show that the improved GOLF learns LP coefficients and reconstructs the voice better than its frame-wise counterparts. Moreover, in our listening test, synthesised outputs from GOLF scored higher in quality ratings than the state-of-the-art differentiable WORLD vocoder.

Differentiable Time-Varying Linear Prediction in the Context of End-to-End Analysis-by-Synthesis

TL;DR

It is shown that the improved GOLF learns LP coefficients and reconstructs the voice better than its frame-wise counterparts, and synthesised outputs from GOLF scored higher in quality ratings than the state-of-the-art differentiable WORLD vocoder.

Abstract

Training the linear prediction (LP) operator end-to-end for audio synthesis in modern deep learning frameworks is slow due to its recursive formulation. In addition, frame-wise approximation as an acceleration method cannot generalise well to test time conditions where the LP is computed sample-wise. Efficient differentiable sample-wise LP for end-to-end training is the key to removing this barrier. We generalise the efficient time-invariant LP implementation from the GOLF vocoder to time-varying cases. Combining this with the classic source-filter model, we show that the improved GOLF learns LP coefficients and reconstructs the voice better than its frame-wise counterparts. Moreover, in our listening test, synthesised outputs from GOLF scored higher in quality ratings than the state-of-the-art differentiable WORLD vocoder.
Paper Structure (19 sections, 10 equations, 3 figures, 1 table)

This paper contains 19 sections, 10 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Harmonic-plus-noise GOLF
  4. Differentiable time-invariant LP
  5. Methodology
  6. Source-filter GOLF
  7. Differentiable time-varying LP
  8. The gradients to $e(t)$
  9. The gradients to $\Tilde{\bf a}(t)$
  10. Experiment
  11. Dataset and training configurations
  12. Speaker-independent encoder
  13. Decoders
  14. Evaluations and discussions
  15. Objective evaluations
  16. ...and 4 more sections

Figures (3)

  • Figure 1: Flow diagram of the proposed experiment. For DDSP, the $G(z)H(z)$ is jointly modelled using an additive synthesiser.
  • Figure 2: The running spectra converted from the encoded LPCs using 0.4 seconds of speech from speaker p360. The rightmost LPCs are computed using the auto-correlation method from SPTK with the same filter order as GOLFs.
  • Figure 3: The average ratings of each speaker with a 95% confidence interval.