Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spatial Voice Conversion: Voice Conversion Preserving Spatial Information and Non-target Signals

Kentaro Seki, Shinnosuke Takamichi, Norihiro Takamune, Yuki Saito, Kanami Imamura, Hiroshi Saruwatari

TL;DR

The baseline approach addresses the gap by integrating blind source separation, voice conversion, and spatial mixing to handle multi-channel waveforms and highlights the fundamental difficulties in balancing these aspects, providing a benchmark for future research in spatial voice conversion.

Abstract

This paper proposes a new task called spatial voice conversion, which aims to convert a target voice while preserving spatial information and non-target signals. Traditional voice conversion methods focus on single-channel waveforms, ignoring the stereo listening experience inherent in human hearing. Our baseline approach addresses this gap by integrating blind source separation (BSS), voice conversion (VC), and spatial mixing to handle multi-channel waveforms. Through experimental evaluations, we organize and identify the key challenges inherent in this task, such as maintaining audio quality and accurately preserving spatial information. Our results highlight the fundamental difficulties in balancing these aspects, providing a benchmark for future research in spatial voice conversion. The proposed method's code is publicly available to encourage further exploration in this domain.

Spatial Voice Conversion: Voice Conversion Preserving Spatial Information and Non-target Signals

TL;DR

The baseline approach addresses the gap by integrating blind source separation, voice conversion, and spatial mixing to handle multi-channel waveforms and highlights the fundamental difficulties in balancing these aspects, providing a benchmark for future research in spatial voice conversion.

Abstract

This paper proposes a new task called spatial voice conversion, which aims to convert a target voice while preserving spatial information and non-target signals. Traditional voice conversion methods focus on single-channel waveforms, ignoring the stereo listening experience inherent in human hearing. Our baseline approach addresses this gap by integrating blind source separation (BSS), voice conversion (VC), and spatial mixing to handle multi-channel waveforms. Through experimental evaluations, we organize and identify the key challenges inherent in this task, such as maintaining audio quality and accurately preserving spatial information. Our results highlight the fundamental difficulties in balancing these aspects, providing a benchmark for future research in spatial voice conversion. The proposed method's code is publicly available to encourage further exploration in this domain.

Paper Structure

This paper contains 17 sections, 5 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Problem definition
  4. Observation process
  5. Our purpose
  6. Proposed method
  7. Blind source separation
  8. Voice conversion
  9. Remixing
  10. Discussion
  11. Experimental evaluations
  12. Data and settings
  13. Results
  14. Acoustic quality
  15. Spatial information
  16. ...and 2 more sections

Figures (3)

  • Figure 1: Spatial voice conversion. It converts target-speaker signal in multi-channel observations while unchanging spatial information and non-target signals.
  • Figure 2: Proposed method procedure: We record mixed audio from multiple speakers using multiple microphones, apply BSS to extract the voice of the BSS-target speaker (the same as the VC-source speaker), apply VC exclusively to this voice, and then output the remixed multi-channel signal.
  • Figure 3: Configurations of speakers and microphones, where "×" and "o" denote speakers and microphones, respectively.