Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Domain-Incremental Continual Learning for Robust and Efficient Keyword Spotting in Resource Constrained Systems

Prakash Dhungana, Sayed Ahmad Salehi

TL;DR

The paper tackles domain shifts in keyword spotting by proposing an on-device domain-incremental continual learning framework that updates a compact quantized model using effective runtime samples and a rehearsal memory. It leverages a dual-input CNN (MFCC and LogMel) with integrated wavelet and spectral denoising to produce robust features, and uses class prototypes to guide pseudo-labeling and sample selection during continual adaptation. Key contributions include a complete update strategy for the quantized model, prototype-based effective-sample determination, and extensive evaluation showing near-clean accuracy on noisy data (above 94% across several environments and down to -10 dB SNR), while maintaining a tiny footprint suitable for embedded hardware. The framework demonstrates that efficient denoising combined with prototype-based continual learning enables autonomous, robust keyword spotting in dynamic, resource-constrained environments, with results suggesting strong potential for real-world TinyML deployment and future multi-class extension.

Abstract

Keyword Spotting (KWS) systems with small footprint models deployed on edge devices face significant accuracy and robustness challenges due to domain shifts caused by varying noise and recording conditions. To address this, we propose a comprehensive framework for continual learning designed to adapt to new domains while maintaining computational efficiency. The proposed pipeline integrates a dual-input Convolutional Neural Network, utilizing both Mel Frequency Cepstral Coefficients (MFCC) and Mel-spectrogram features, supported by a multi-stage denoising process, involving discrete wavelet transform and spectral subtraction techniques, plus model and prototype update blocks. Unlike prior methods that restrict updates to specific layers, our approach updates the complete quantized model, made possible due to compact model architecture. A subset of input samples are selected during runtime using class prototypes and confidence-driven filtering, which are then pseudo-labeled and combined with rehearsal buffer for incremental model retraining. Experimental results on noisy test dataset demonstrate the framework's effectiveness, achieving 99.63\% accuracy on clean data and maintaining robust performance (exceeding 94\% accuracy) across diverse noisy environments, even at -10 dB Signal-to-Noise Ratio. The proposed framework work confirms that integrating efficient denoising with prototype-based continual learning enables KWS models to operate autonomously and robustly in resource-constrained, dynamic environments.

Domain-Incremental Continual Learning for Robust and Efficient Keyword Spotting in Resource Constrained Systems

TL;DR

The paper tackles domain shifts in keyword spotting by proposing an on-device domain-incremental continual learning framework that updates a compact quantized model using effective runtime samples and a rehearsal memory. It leverages a dual-input CNN (MFCC and LogMel) with integrated wavelet and spectral denoising to produce robust features, and uses class prototypes to guide pseudo-labeling and sample selection during continual adaptation. Key contributions include a complete update strategy for the quantized model, prototype-based effective-sample determination, and extensive evaluation showing near-clean accuracy on noisy data (above 94% across several environments and down to -10 dB SNR), while maintaining a tiny footprint suitable for embedded hardware. The framework demonstrates that efficient denoising combined with prototype-based continual learning enables autonomous, robust keyword spotting in dynamic, resource-constrained environments, with results suggesting strong potential for real-world TinyML deployment and future multi-class extension.

Abstract

Keyword Spotting (KWS) systems with small footprint models deployed on edge devices face significant accuracy and robustness challenges due to domain shifts caused by varying noise and recording conditions. To address this, we propose a comprehensive framework for continual learning designed to adapt to new domains while maintaining computational efficiency. The proposed pipeline integrates a dual-input Convolutional Neural Network, utilizing both Mel Frequency Cepstral Coefficients (MFCC) and Mel-spectrogram features, supported by a multi-stage denoising process, involving discrete wavelet transform and spectral subtraction techniques, plus model and prototype update blocks. Unlike prior methods that restrict updates to specific layers, our approach updates the complete quantized model, made possible due to compact model architecture. A subset of input samples are selected during runtime using class prototypes and confidence-driven filtering, which are then pseudo-labeled and combined with rehearsal buffer for incremental model retraining. Experimental results on noisy test dataset demonstrate the framework's effectiveness, achieving 99.63\% accuracy on clean data and maintaining robust performance (exceeding 94\% accuracy) across diverse noisy environments, even at -10 dB Signal-to-Noise Ratio. The proposed framework work confirms that integrating efficient denoising with prototype-based continual learning enables KWS models to operate autonomously and robustly in resource-constrained, dynamic environments.
Paper Structure (30 sections, 5 equations, 8 figures, 1 table, 2 algorithms)

This paper contains 30 sections, 5 equations, 8 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. On device deep learning approaches
  4. Continual learning for domain adaptation
  5. Proposed Methodology
  6. KWS Pipeline
  7. Wavelet Denoising
  8. Feature Extraction
  9. Spectral Denoising
  10. Single feature map model architecture
  11. Dual feature map model architecture
  12. Effective samples Determination
  13. Class Prototypes
  14. Effective Samples
  15. Continual Learning
  16. ...and 15 more sections

Figures (8)

  • Figure 1: Overview of proposed framework for continual learning.
  • Figure 2: An example of denoising masks generated for MFCC (top row) and Mel-Spectrogaram (bottom row).
  • Figure 3: Proposed classifier model (single input) architecture for continual learning.
  • Figure 4: Proposed classifier model (dual input) architecture for continual learning.
  • Figure 5: Determination of effective samples to be included in training mini-batch for continual learning.
  • ...and 3 more figures