Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Talking Robot: Distortion-Robust Acoustic Models for Robot-Robot Communication

Hanlong Li, Karishma Kamalahasan, Jiahui Li, Kazuhiro Nakadai, Shreyas Kousik

TL;DR

Artoo, a learned acoustic communication system for robots that replaces hand-designed signal processing with end-to-end co-trained neural networks, is presented, suitable for deployment on resource-constrained robotic platforms.

Abstract

We present Artoo, a learned acoustic communication system for robots that replaces hand-designed signal processing with end-to-end co-trained neural networks. Our system pairs a lightweight text-to-speech (TTS) transmitter (1.18M parameters) with a conformer-based automatic speech recognition (ASR) receiver (938K parameters), jointly optimized through a differentiable channel. Unlike human speech, robot-to-robot communication is paralinguistics-free: the system need not preserve timbre, prosody, or naturalness, only maximize decoding accuracy under channel distortion. Through a three-phase co-training curriculum, the TTS transmitter learns to produce distortion-robust acoustic encodings that surpass the baseline under noise, achieving 8.3% CER at 0 dB SNR. The entire system requires only 2.1M parameters (8.4 MB) and runs in under 13 ms end-to-end on a CPU, making it suitable for deployment on resource-constrained robotic platforms.

The Talking Robot: Distortion-Robust Acoustic Models for Robot-Robot Communication

TL;DR

Artoo, a learned acoustic communication system for robots that replaces hand-designed signal processing with end-to-end co-trained neural networks, is presented, suitable for deployment on resource-constrained robotic platforms.

Abstract

We present Artoo, a learned acoustic communication system for robots that replaces hand-designed signal processing with end-to-end co-trained neural networks. Our system pairs a lightweight text-to-speech (TTS) transmitter (1.18M parameters) with a conformer-based automatic speech recognition (ASR) receiver (938K parameters), jointly optimized through a differentiable channel. Unlike human speech, robot-to-robot communication is paralinguistics-free: the system need not preserve timbre, prosody, or naturalness, only maximize decoding accuracy under channel distortion. Through a three-phase co-training curriculum, the TTS transmitter learns to produce distortion-robust acoustic encodings that surpass the baseline under noise, achieving 8.3% CER at 0 dB SNR. The entire system requires only 2.1M parameters (8.4 MB) and runs in under 13 ms end-to-end on a CPU, making it suitable for deployment on resource-constrained robotic platforms.
Paper Structure (45 sections, 7 equations, 2 figures, 14 tables)

This paper contains 45 sections, 7 equations, 2 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Contributions
  3. Paper Organization
  4. Related Work
  5. Acoustic Data Communication
  6. Learned Communication Systems
  7. Neural Text-to-Speech and ASR
  8. Artoo System Architecture
  9. System Overview
  10. Artoo Deployment Pipeline
  11. Vocabulary/Tokens
  12. Pretraining Architecture
  13. TTS Transmitter
  14. Griffin-Lim Vocoder
  15. ASR Receiver
  16. ...and 30 more sections

Figures (2)

  • Figure 1: We propose Artoo: a text-to-speech (TTS) and automatic speech recognition (ASR) pipeline for robot-robot communication. The goal of our method is to enable robots to communicate in a way that is robust to environmental noise; our key insight is to relax the need for TTS and ASR models to replicate human speech.
  • Figure 2: Illustration of the real-world over-the-air deployment setup used for hardware evaluation.