Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Lightweight and Real-Time Binaural Speech Enhancement Model with Spatial Cues Preservation

Jingyuan Wang, Jie Zhang, Shihao Chen, Miao Sun

TL;DR

This work tackles real-time binaural speech enhancement with the dual goals of noise reduction and preservation of spatial cues. It introduces LBCCN, a lightweight complex convolutional network built with a band-compressed feature extractor, dual-path modeling, and RATF-based binaural signal predictors to jointly enhance speech and maintain binaural spatial information. Key contributions include processing only low-frequency bands to reduce computation, explicit estimation of target RATFs for improved SCP, and a loss function balancing NR and SCP across selected TF bands. On a synthesized spatialized binaural dataset, LBCCN achieves competitive or superior intelligibility (MBSTOI) and spatial-cue preservation with substantially lower model size and real-time cost, highlighting its potential for low-resource, latency-sensitive listening devices.

Abstract

Binaural speech enhancement (BSE) aims to jointly improve the speech quality and intelligibility of noisy signals received by hearing devices and preserve the spatial cues of the target for natural listening. Existing methods often suffer from the compromise between noise reduction (NR) capacity and spatial cues preservation (SCP) accuracy and a high computational demand in complex acoustic scenes. In this work, we present a learning-based lightweight binaural complex convolutional network (LBCCN), which excels in NR by filtering low-frequency bands and keeping the rest. Additionally, our approach explicitly incorporates the estimation of interchannel relative acoustic transfer function to ensure the spatial cues fidelity and speech clarity. Results show that the proposed LBCCN can achieve a comparable NR performance to state-of-the-art methods under fixed-speaker conditions, but with a much lower computational cost and a certain degree of SCP capability. The reproducible code and audio examples are available at https://github.com/jywanng/LBCCN.

A Lightweight and Real-Time Binaural Speech Enhancement Model with Spatial Cues Preservation

TL;DR

This work tackles real-time binaural speech enhancement with the dual goals of noise reduction and preservation of spatial cues. It introduces LBCCN, a lightweight complex convolutional network built with a band-compressed feature extractor, dual-path modeling, and RATF-based binaural signal predictors to jointly enhance speech and maintain binaural spatial information. Key contributions include processing only low-frequency bands to reduce computation, explicit estimation of target RATFs for improved SCP, and a loss function balancing NR and SCP across selected TF bands. On a synthesized spatialized binaural dataset, LBCCN achieves competitive or superior intelligibility (MBSTOI) and spatial-cue preservation with substantially lower model size and real-time cost, highlighting its potential for low-resource, latency-sensitive listening devices.

Abstract

Binaural speech enhancement (BSE) aims to jointly improve the speech quality and intelligibility of noisy signals received by hearing devices and preserve the spatial cues of the target for natural listening. Existing methods often suffer from the compromise between noise reduction (NR) capacity and spatial cues preservation (SCP) accuracy and a high computational demand in complex acoustic scenes. In this work, we present a learning-based lightweight binaural complex convolutional network (LBCCN), which excels in NR by filtering low-frequency bands and keeping the rest. Additionally, our approach explicitly incorporates the estimation of interchannel relative acoustic transfer function to ensure the spatial cues fidelity and speech clarity. Results show that the proposed LBCCN can achieve a comparable NR performance to state-of-the-art methods under fixed-speaker conditions, but with a much lower computational cost and a certain degree of SCP capability. The reproducible code and audio examples are available at https://github.com/jywanng/LBCCN.
Paper Structure (10 sections, 7 equations, 2 figures, 3 tables)

This paper contains 10 sections, 7 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Proposed LBCCN Method
  3. Band-Compressed Feature Extractor
  4. Dual-Path Modeling
  5. Binaural Signal Predictors
  6. Loss Function
  7. Experiments
  8. Experimental Setup
  9. Experimental Results
  10. Conclusion

Figures (2)

  • Figure 1: The proposed LBCCN BSE model, which mainly consists of band-compressed feature extractor (operates on the lower-$Q$ frequency bands), dual-path modeling and signal predictors. The LightConv1D operates on the frequency dimension, and LightConv2D on both time and frequency dimensions.
  • Figure 2: The impact of the signal and noise losses weight $k$ in (5).