UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generation

TL;DR

UnivNet introduces a neural vocoder equipped with a multi-resolution spectrogram discriminator to address over-smoothing when using full-band inputs. By combining MRSD with a multi-period waveform discriminator and a spectrogram-based auxiliary loss, the model achieves high-fidelity, real-time waveform synthesis and strong robustness to unseen speakers. Experiments on LibriTTS and LJSpeech show superior objective and subjective performance versus GAN-based baselines, and rapid adaptation in text-to-speech through fine-tuning. The work demonstrates that multi-resolution spectral supervision can substantially improve the quality of full-band vocoding, with promising implications for universal multi-speaker TTS systems.

Abstract

Most neural vocoders employ band-limited mel-spectrograms to generate waveforms. If full-band spectral features are used as the input, the vocoder can be provided with as much acoustic information as possible. However, in some models employing full-band mel-spectrograms, an over-smoothing problem occurs as part of which non-sharp spectrograms are generated. To address this problem, we propose UnivNet, a neural vocoder that synthesizes high-fidelity waveforms in real time. Inspired by works in the field of voice activity detection, we added a multi-resolution spectrogram discriminator that employs multiple linear spectrogram magnitudes computed using various parameter sets. Using full-band mel-spectrograms as input, we expect to generate high-resolution signals by adding a discriminator that employs spectrograms of multiple resolutions as the input. In an evaluation on a dataset containing information on hundreds of speakers, UnivNet obtained the best objective and subjective results among competing models for both seen and unseen speakers. These results, including the best subjective score for text-to-speech, demonstrate the potential for fast adaptation to new speakers without a need for training from scratch.

Figures (3)

• Figure 1: UnivNet architecture. ‘$\mathrm{STFT\ }\#m$’ denotes the process of computing a linear spectrogram magnitude using the $m$-th STFT parameter set. ‘$\mathrm{reshape2d(}p\mathrm{)}$’ denotes the process of reshaping a 1-D signal of length $T$ to a 2-D signal of height $T/p$ and width $p$.
• Figure 2: Details of UnivNet as used in the experiments. The first two numbers in parentheses for each layer indicate the channel size and kernel size, respectively.
• Figure 3: Spectrograms of the generated audio clips.