Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Lightweight Fourier-based Network for Binaural Speech Enhancement with Spatial Cue Preservation

Xikun Lu, Yujian Ma, Xianquan Jiang, Xuelong Wang, Jinqiu Sang

TL;DR

This work addresses the challenge of achieving high-fidelity binaural speech enhancement on resource-constrained devices while preserving spatial cues. It introduces GAF-Net, a lightweight complex-valued network that combines a dual-feature encoder, a Global Adaptive Fourier Modulator backbone for global temporal modeling in the Fourier domain, and a Dynamic Refinement Gate to suppress artifacts. The approach delivers competitive MBSTOI and superior ILD/IPD preservation with only 129K parameters and 2.79 GMACs, demonstrating a favorable performance-efficiency balance. Ablation studies confirm the key roles of dual-feature encoding, the Fourier-based backbone, and the gating mechanism in balancing denoising with spatial fidelity. While validated in anechoic conditions, the work highlights future extensions to reverberant environments and broader multichannel audio tasks.

Abstract

Binaural speech enhancement faces a severe trade-off challenge, where state-of-the-art performance is achieved by computationally intensive architectures, while lightweight solutions often come at the cost of significant performance degradation. To bridge this gap, we propose the Global Adaptive Fourier Network (GAF-Net), a lightweight deep complex network that aims to establish a balance between performance and computational efficiency. The GAF-Net architecture consists of three components. First, a dual-feature encoder combining short-time Fourier transform and gammatone features enhances the robustness of acoustic representation. Second, a channel-independent globally adaptive Fourier modulator efficiently captures long-term temporal dependencies while preserving the spatial cues. Finally, a dynamic gating mechanism is implemented to reduce processing artifacts. Experimental results show that GAF-Net achieves competitive performance, particularly in terms of binaural cues (ILD and IPD error) and objective intelligibility (MBSTOI), with fewer parameters and computational cost. These results confirm that GAF-Net provides a feasible way to achieve high-fidelity binaural processing on resource-constrained devices.

A Lightweight Fourier-based Network for Binaural Speech Enhancement with Spatial Cue Preservation

TL;DR

This work addresses the challenge of achieving high-fidelity binaural speech enhancement on resource-constrained devices while preserving spatial cues. It introduces GAF-Net, a lightweight complex-valued network that combines a dual-feature encoder, a Global Adaptive Fourier Modulator backbone for global temporal modeling in the Fourier domain, and a Dynamic Refinement Gate to suppress artifacts. The approach delivers competitive MBSTOI and superior ILD/IPD preservation with only 129K parameters and 2.79 GMACs, demonstrating a favorable performance-efficiency balance. Ablation studies confirm the key roles of dual-feature encoding, the Fourier-based backbone, and the gating mechanism in balancing denoising with spatial fidelity. While validated in anechoic conditions, the work highlights future extensions to reverberant environments and broader multichannel audio tasks.

Abstract

Binaural speech enhancement faces a severe trade-off challenge, where state-of-the-art performance is achieved by computationally intensive architectures, while lightweight solutions often come at the cost of significant performance degradation. To bridge this gap, we propose the Global Adaptive Fourier Network (GAF-Net), a lightweight deep complex network that aims to establish a balance between performance and computational efficiency. The GAF-Net architecture consists of three components. First, a dual-feature encoder combining short-time Fourier transform and gammatone features enhances the robustness of acoustic representation. Second, a channel-independent globally adaptive Fourier modulator efficiently captures long-term temporal dependencies while preserving the spatial cues. Finally, a dynamic gating mechanism is implemented to reduce processing artifacts. Experimental results show that GAF-Net achieves competitive performance, particularly in terms of binaural cues (ILD and IPD error) and objective intelligibility (MBSTOI), with fewer parameters and computational cost. These results confirm that GAF-Net provides a feasible way to achieve high-fidelity binaural processing on resource-constrained devices.

Paper Structure

This paper contains 13 sections, 9 equations, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Proposed method
  3. Dual-Feature Encoding and Fusion
  4. Global Adaptive Fourier Modulator
  5. Binaural Decoding and Dynamic Refinement Gate
  6. Loss Function
  7. Experiments
  8. Datasets
  9. Implementation Details
  10. Evaluation Metrics
  11. Results and discussion
  12. Conclusion
  13. ACKNOWLEDGEMENTS

Figures (1)

  • Figure 1: The proposed GAF-Net model, which mainly consists of dual-feature encoding and fusion, global adaptive Fourier modulator, and dynamic refinement gate. Different modules are marked with different colors.