Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Transforming NLU with Babylon: A Case Study in Development of Real-time, Edge-Efficient, Multi-Intent Translation System for Automated Drive-Thru Ordering

Mostafa Varzaneh, Pooja Voladoddi, Tanmay Bakshi, Uma Gunturi

TL;DR

B Babylon is introduced, a transformer-based architecture that tackles NLU as an intent translation task, converting natural language inputs into sequences of regular language units ('transcodes') that encode both intents and slot information in a single dialogue turn.

Abstract

Real-time conversational AI agents face challenges in performing Natural Language Understanding (NLU) in dynamic, outdoor environments like automated drive-thru systems. These settings require NLU models to handle background noise, diverse accents, and multi-intent queries while operating under strict latency and memory constraints on edge devices. Additionally, robustness to errors from upstream Automatic Speech Recognition (ASR) is crucial, as ASR outputs in these environments are often noisy. We introduce Babylon, a transformer-based architecture that tackles NLU as an intent translation task, converting natural language inputs into sequences of regular language units ('transcodes') that encode both intents and slot information. This formulation allows Babylon to manage multi-intent scenarios in a single dialogue turn. Furthermore, Babylon incorporates an LSTM-based token pooling mechanism to preprocess phoneme sequences, reducing input length and optimizing for low-latency, low-memory edge deployment. This also helps mitigate inaccuracies in ASR outputs, enhancing system robustness. While this work focuses on drive-thru ordering, Babylon's design extends to similar noise-prone scenarios, for e.g. ticketing kiosks. Our experiments show that Babylon achieves significantly better accuracy-latency-memory footprint trade-offs over typically employed NMT models like Flan-T5 and BART, demonstrating its effectiveness for real-time NLU in edge deployment settings.

Transforming NLU with Babylon: A Case Study in Development of Real-time, Edge-Efficient, Multi-Intent Translation System for Automated Drive-Thru Ordering

TL;DR

B Babylon is introduced, a transformer-based architecture that tackles NLU as an intent translation task, converting natural language inputs into sequences of regular language units ('transcodes') that encode both intents and slot information in a single dialogue turn.

Abstract

Real-time conversational AI agents face challenges in performing Natural Language Understanding (NLU) in dynamic, outdoor environments like automated drive-thru systems. These settings require NLU models to handle background noise, diverse accents, and multi-intent queries while operating under strict latency and memory constraints on edge devices. Additionally, robustness to errors from upstream Automatic Speech Recognition (ASR) is crucial, as ASR outputs in these environments are often noisy. We introduce Babylon, a transformer-based architecture that tackles NLU as an intent translation task, converting natural language inputs into sequences of regular language units ('transcodes') that encode both intents and slot information. This formulation allows Babylon to manage multi-intent scenarios in a single dialogue turn. Furthermore, Babylon incorporates an LSTM-based token pooling mechanism to preprocess phoneme sequences, reducing input length and optimizing for low-latency, low-memory edge deployment. This also helps mitigate inaccuracies in ASR outputs, enhancing system robustness. While this work focuses on drive-thru ordering, Babylon's design extends to similar noise-prone scenarios, for e.g. ticketing kiosks. Our experiments show that Babylon achieves significantly better accuracy-latency-memory footprint trade-offs over typically employed NMT models like Flan-T5 and BART, demonstrating its effectiveness for real-time NLU in edge deployment settings.

Paper Structure

This paper contains 41 sections, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. System Description
  3. Related Work
  4. Conventional NLU approaches
  5. Neural Machine Translation for Multi-Intent Detection
  6. Limitations of vanilla Transformers
  7. Babylon Architecture
  8. LSTM, the hero!
  9. LSTM as neural rolling memory:
  10. LSTM for keeping temporal information:
  11. LSTM as regularizer:
  12. Downsampling by token pooling
  13. Experiments
  14. Hardware Setup
  15. Dataset
  16. ...and 26 more sections

Figures (3)

  • Figure 1: The input to our NLU model for this customer order is a sequence of phonemes. The output generated by the NLU model post translation [1 | coffee | item 0 | sugar| sub_item | 0 | cream | sub_item | add] as seen in the figure, is a sequence of output transcodes and represents the customer intent to add a coffee to their order and remove sugar and cream from it.
  • Figure 2: Illustration of our overall system. We re-purpose the NLU task of intent classification and slot filling into a translation task. We translate from audio units (phonemes) to intent language units (transcodes).
  • Figure 3: Architecture of Babylon. Blocks highlighted in yellow represent the LSTM and Token pooling layers added prior to the original vaswani2017attention transformer architecture.