Discriminating real and synthetic super-resolved audio samples using embedding-based classifiers

TL;DR

The paper tackles whether synthetic wideband audio from ADSR models matches real audio at the distributional level, beyond perceptual quality. It probes separability in multiple embedding spaces by training linear classifiers on frozen discriminator features from MU-GAN and on external embeddings (OpenL3 and log-Mel), across 4→16 kHz and 16→48 kHz upsampling. Findings show near-perfect separation for external embeddings and significant separability for discriminator features, even when perceptual metrics and MOS-like scores look strong, highlighting a gap between perceptual realism and distributional fidelity. The results hold across VCTK and FMA-small datasets and include diffusion-based methods, underscoring the need for distribution-aware evaluation in ADSR research.

Abstract

Generative adversarial networks (GANs) and diffusion models have recently achieved state-of-the-art performance in audio super-resolution (ADSR), producing perceptually convincing wideband audio from narrowband inputs. However, existing evaluations primarily rely on signal-level or perceptual metrics, leaving open the question of how closely the distributions of synthetic super-resolved and real wideband audio match. Here we address this problem by analyzing the separability of real and super-resolved audio in various embedding spaces. We consider both middle-band ($4\to 16$~kHz) and full-band ($16\to 48$~kHz) upsampling tasks for speech and music, training linear classifiers to distinguish real from synthetic samples based on multiple types of audio embeddings. Comparisons with objective metrics and subjective listening tests reveal that embedding-based classifiers achieve near-perfect separation, even when the generated audio attains high perceptual quality and state-of-the-art metric scores. This behavior is consistent across datasets and models, including recent diffusion-based approaches, highlighting a persistent gap between perceptual quality and true distributional fidelity in ADSR models.

Figures (10)

• Figure 1: Feature extraction and 'real'/'fake' classification task. (a) Discriminator-based classifier uses internal discriminator features produced by its pre-fully connected (pre-FC) layers with frozen weights. (b) External classifier operating on independent features extracted by a separate network, enabling an analysis of potential representation bias between the two classifiers.
• Figure 2: Listener scores for different ADSR methods. MUSHRA scores for listening tests across different conditions WB, MU-GAN, AudioUnet, HiFi-GAN, LP 3.5kHz, Spline-Up 7kHz. (a) Inter-quartile range (IQR), medians, and mean values by green triangles. (b) Mean scores with error bars representing $95\%$ confidence intervals.
• Figure 3: MU-GAN 16 kHz, Discriminator
• Figure 4: MU-GAN 16 kHz, OpenL3
• Figure 5: AudioUnet 16 kHz, Discriminator