Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EBEN: Extreme bandwidth extension network applied to speech signals captured with noise-resilient body-conduction microphones

Julien Hauret, Thomas Joubaud, Véronique Zimpfer, Éric Bavu

TL;DR

Body-conduction microphone speech suffers from ambient noise and reduced bandwidth, limiting intelligibility. EBEN introduces a GAN-based extreme bandwidth extension network that uses an $M$-band PQMF decomposition and a U‑Net‑like generator to reconstruct wideband speech in real time, aided by a multiscale ensemble of subband discriminators. The model balances adversarial and reconstruction losses to preserve realism while recovering missing high-frequency content. Experiments on simulated in-ear degradations show competitive objective metrics and strong subjective scores, with a lightweight, edge-friendly architecture suitable for real-time deployment and potential public dataset release.

Abstract

In this paper, we present Extreme Bandwidth Extension Network (EBEN), a Generative Adversarial network (GAN) that enhances audio measured with body-conduction microphones. This type of capture equipment suppresses ambient noise at the expense of speech bandwidth, thereby requiring signal enhancement techniques to recover the wideband speech signal. EBEN leverages a multiband decomposition of the raw captured speech to decrease the data time-domain dimensions, and give better control over the full-band signal. This multiband representation is fed to a U-Net-like model, which adopts a combination of feature and adversarial losses to recover an enhanced audio signal. We also benefit from this original representation in the proposed discriminator architecture. Our approach can achieve state-of-the-art results with a lightweight generator and real-time compatible operation.

EBEN: Extreme bandwidth extension network applied to speech signals captured with noise-resilient body-conduction microphones

TL;DR

Body-conduction microphone speech suffers from ambient noise and reduced bandwidth, limiting intelligibility. EBEN introduces a GAN-based extreme bandwidth extension network that uses an -band PQMF decomposition and a U‑Net‑like generator to reconstruct wideband speech in real time, aided by a multiscale ensemble of subband discriminators. The model balances adversarial and reconstruction losses to preserve realism while recovering missing high-frequency content. Experiments on simulated in-ear degradations show competitive objective metrics and strong subjective scores, with a lightweight, edge-friendly architecture suitable for real-time deployment and potential public dataset release.

Abstract

In this paper, we present Extreme Bandwidth Extension Network (EBEN), a Generative Adversarial network (GAN) that enhances audio measured with body-conduction microphones. This type of capture equipment suppresses ambient noise at the expense of speech bandwidth, thereby requiring signal enhancement techniques to recover the wideband speech signal. EBEN leverages a multiband decomposition of the raw captured speech to decrease the data time-domain dimensions, and give better control over the full-band signal. This multiband representation is fed to a U-Net-like model, which adopts a combination of feature and adversarial losses to recover an enhanced audio signal. We also benefit from this original representation in the proposed discriminator architecture. Our approach can achieve state-of-the-art results with a lightweight generator and real-time compatible operation.
Paper Structure (11 sections, 3 equations, 5 figures, 1 table)

This paper contains 11 sections, 3 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methods
  3. Degradation estimation
  4. Pseudo Quadrature Mirror Filter formalism
  5. Model architecture
  6. Loss functions
  7. Experiments
  8. Simulation of the dataset
  9. Objective evaluation
  10. Subjective evaluation
  11. Discussion and Future Work

Figures (5)

  • Figure 1: Degradation analysis of the in-ear transducer
  • Figure 2: Frequency response of a PQMF filter bank
  • Figure 3: PQMF Analysis and Synthesis : block-diagram
  • Figure 4: Overall architecture represented for $M=4$, one band with voice content fed to generator, others to PQMF discriminators
  • Figure 5: Spectrograms of various bandwidth extension models sandwiched by the simulated in-ear and the reference signals.