A 14uJ/Decision Keyword Spotting Accelerator with In-SRAM-Computing and On Chip Learning for Customization

TL;DR

The paper tackles the challenge of energy-efficient keyword spotting on edge devices by proposing a SRAM-based in-memory computing accelerator with on-chip learning for user customization. It addresses IMC non-idealities through bias compensation, fine-tuning, and hardware-aware training techniques including error scaling, small gradient accumulation, and random gradient prediction, achieving $14\,\mu J$ per decision and accuracy up to $96.71\%$ with customization. The architecture combines digital first/final layers with an IMC core, enabling high energy efficiency (up to $68$ TOPS/W) at 28 nm, while maintaining on-chip adaptation capabilities. The results show substantial improvements over state-of-the-art in both energy efficiency and personalization support, making edge-scale KWS more practical for privacy-preserving, always-on devices.

Abstract

Keyword spotting has gained popularity as a natural way to interact with consumer devices in recent years. However, because of its always-on nature and the variety of speech, it necessitates a low-power design as well as user customization. This paper describes a low-power, energy-efficient keyword spotting accelerator with SRAM based in-memory computing (IMC) and on-chip learning for user customization. However, IMC is constrained by macro size, limited precision, and non-ideal effects. To address the issues mentioned above, this paper proposes bias compensation and fine-tuning using an IMC-aware model design. Furthermore, because learning with low-precision edge devices results in zero error and gradient values due to quantization, this paper proposes error scaling and small gradient accumulation to achieve the same accuracy as ideal model training. The simulation results show that with user customization, we can recover the accuracy loss from 51.08\% to 89.76\% with compensation and fine-tuning and further improve to 96.71\% with customization. The chip implementation can successfully run the model with only 14$uJ$ per decision. When compared to the state-of-the-art works, the presented design has higher energy efficiency with additional on-chip model customization capabilities for higher accuracy.

Figures (18)

• Figure 1: The proposed IMC aware KWS model. The number in the block means output channel number, kernel size, pooling size from left to right.
• Figure 2: A learnable offset before activation function to change the binarized threshold of different layer. For a positive offset(top right), more feature will be 1 after activation, for a negative offset(bottom right), more feature will be 0 after activation.
• Figure 3: Trained offset of each layer with initial zero value.
• Figure 4: The Gradient distribution of our KWS model on the personal dataset before quantize(top) and after quantize(bottom).
• Figure 5: (a) Weight update with random gradient prediction, where the yellow arrow means the random direction. (b) Weight update without random gradient prediction.