Single-Channel Robot Ego-Speech Filtering during Human-Robot Interaction

TL;DR

This work tackles the problem of maintaining an open microphone during human–robot interaction by extracting the human target speech from a single-channel mix that includes Pepper's ego speech and fan noise. It compares a signal-processing pipeline based on spectral subtraction with a CRNN-based TSE model, using a manufactured Pepper/Librispeech dataset and evaluation via Word Error Rate (WER) and SI-SDR, along with computing time. The key findings are that the spectral-subtraction approach without post-filtering yields the best WER under low reverberation, while the CRNN approach provides robustness to reverberation and the best SI-SDR, with baseline methods performing poorly. These results suggest that natural HRI with open microphones is feasible under favorable acoustic conditions and provide guidance for future improvements, including dereverberation, larger datasets, and end-to-end ASR integration.

Abstract

In this paper, we study how well human speech can automatically be filtered when this overlaps with the voice and fan noise of a social robot, Pepper. We ultimately aim for an HRI scenario where the microphone can remain open when the robot is speaking, enabling a more natural turn-taking scheme where the human can interrupt the robot. To respond appropriately, the robot would need to understand what the interlocutor said in the overlapping part of the speech, which can be accomplished by target speech extraction (TSE). To investigate how well TSE can be accomplished in the context of the popular social robot Pepper, we set out to manufacture a datase composed of a mixture of recorded speech of Pepper itself, its fan noise (which is close to the microphones), and human speech as recorded by the Pepper microphone, in a room with low reverberation and high reverberation. Comparing a signal processing approach, with and without post-filtering, and a convolutional recurrent neural network (CRNN) approach to a state-of-the-art speaker identification-based TSE model, we found that the signal processing approach without post-filtering yielded the best performance in terms of Word Error Rate on the overlapping speech signals with low reverberation, while the CRNN approach is more robust for reverberation. These results show that estimating the human voice in overlapping speech with a robot is possible in real-life application, provided that the room reverberation is low and the human speech has a high volume or high pitch.

Figures (10)

• Figure 1: Spectrogram of the recorded speech signal and the corresponding speech signal played by the robot.
• Figure 2: The recorded overlapping speech signal on time domain.
• Figure 3: The illustration of the generic application scenario.
• Figure 4: Signal Processing Pipeline. In the pipeline, the input texture is sent to on cloud text-to-speech API using SIC framework.
• Figure 5: The spectrogram of different signals at time $T_0$.