Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Endpointing with Deep Contextual Multi-armed Bandits

Do June Min, Andreas Stolcke, Anirudh Raju, Colin Vaz, Di He, Venkatesh Ravichandran, Viet Anh Trinh

TL;DR

The paper addresses the inefficiency of static endpointing configurations and offline tuning by introducing an online adaptive endpointing framework based on a deep contextual CMAB. By learning from reward signals that combine latency and cutoff outcomes, and without requiring ground-truth labels, the approach leverages target utterance audio and hypothesis features to decide between a standard and a relaxed endpointing configuration on a per-utterance basis. Findings show that audio and partial hypothesis information are highly informative, with early-cutoff reductions achievable using only a portion of the utterance and with little to no degradation in latency; online CMAB models can approach or match offline supervised performance under certain conditions. This work demonstrates the practical viability of online, reward-driven endpointing in production dialog systems, reducing manual tuning and enabling responsive, low-latency interactions.

Abstract

Current endpointing (EP) solutions learn in a supervised framework, which does not allow the model to incorporate feedback and improve in an online setting. Also, it is a common practice to utilize costly grid-search to find the best configuration for an endpointing model. In this paper, we aim to provide a solution for adaptive endpointing by proposing an efficient method for choosing an optimal endpointing configuration given utterance-level audio features in an online setting, while avoiding hyperparameter grid-search. Our method does not require ground truth labels, and only uses online learning from reward signals without requiring annotated labels. Specifically, we propose a deep contextual multi-armed bandit-based approach, which combines the representational power of neural networks with the action exploration behavior of Thompson modeling algorithms. We compare our approach to several baselines, and show that our deep bandit models also succeed in reducing early cutoff errors while maintaining low latency.

Adaptive Endpointing with Deep Contextual Multi-armed Bandits

TL;DR

The paper addresses the inefficiency of static endpointing configurations and offline tuning by introducing an online adaptive endpointing framework based on a deep contextual CMAB. By learning from reward signals that combine latency and cutoff outcomes, and without requiring ground-truth labels, the approach leverages target utterance audio and hypothesis features to decide between a standard and a relaxed endpointing configuration on a per-utterance basis. Findings show that audio and partial hypothesis information are highly informative, with early-cutoff reductions achievable using only a portion of the utterance and with little to no degradation in latency; online CMAB models can approach or match offline supervised performance under certain conditions. This work demonstrates the practical viability of online, reward-driven endpointing in production dialog systems, reducing manual tuning and enabling responsive, low-latency interactions.

Abstract

Current endpointing (EP) solutions learn in a supervised framework, which does not allow the model to incorporate feedback and improve in an online setting. Also, it is a common practice to utilize costly grid-search to find the best configuration for an endpointing model. In this paper, we aim to provide a solution for adaptive endpointing by proposing an efficient method for choosing an optimal endpointing configuration given utterance-level audio features in an online setting, while avoiding hyperparameter grid-search. Our method does not require ground truth labels, and only uses online learning from reward signals without requiring annotated labels. Specifically, we propose a deep contextual multi-armed bandit-based approach, which combines the representational power of neural networks with the action exploration behavior of Thompson modeling algorithms. We compare our approach to several baselines, and show that our deep bandit models also succeed in reducing early cutoff errors while maintaining low latency.
Paper Structure (12 sections, 4 figures, 2 tables, 1 algorithm)

This paper contains 12 sections, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Task & Methodology
  3. Task: Adaptive endpointing
  4. Dataset
  5. System Architecture & Features
  6. Deep contextual multi-armed bandit (Deep CMAB) algorithm
  7. Experiments
  8. Experiment setup
  9. Which features are most useful?
  10. How much information do we need?
  11. Can an online contextual bandit model be used in place of an offline-trained model?
  12. Conclusions

Figures (4)

  • Figure 1: An overview of our adaptive endpointing with deep contextual multi-armed bandit (CMAB).
  • Figure 2: Shared model architecture for the supervised classifier and deep contextual bandit models. For the bandit models, the classification layer is replaced by a reward predictor for each action.
  • Figure 3: Comparing hypothesis features with varying portions on cutoff-latency tradeoff curves.
  • Figure 4: Comparing supervised and bandit models on first segment input features.