TinySV: Speaker Verification in TinyML with On-device Learning

TL;DR

A new type of adaptive TinyML solution that can be used in tasks, such as the presented Tiny Speaker Verification (TinySV), that require to be tackled with an on-device learning algorithm.

Abstract

TinyML is a novel area of machine learning that gained huge momentum in the last few years thanks to the ability to execute machine learning algorithms on tiny devices (such as Internet-of-Things or embedded systems). Interestingly, research in this area focused on the efficient execution of the inference phase of TinyML models on tiny devices, while very few solutions for on-device learning of TinyML models are available in the literature due to the relevant overhead introduced by the learning algorithms. The aim of this paper is to introduce a new type of adaptive TinyML solution that can be used in tasks, such as the presented \textit{Tiny Speaker Verification} (TinySV), that require to be tackled with an on-device learning algorithm. Achieving this goal required (i) reducing the memory and computational demand of TinyML learning algorithms, and (ii) designing a TinyML learning algorithm operating with few and possibly unlabelled training data. The proposed TinySV solution relies on a two-layer hierarchical TinyML solution comprising Keyword Spotting and Adaptive Speaker Verification module. We evaluated the effectiveness and efficiency of the proposed TinySV solution on a dataset collected expressly for the task and tested the proposed solution on a real-world IoT device (Infineon PSoC 62S2 Wi-Fi BT Pioneer Kit).

Figures (7)

• Figure 1: Examples of the use case, in which $k$ = "Sheila" and the enrolled speaker $S_E$ is Bob.
• Figure 2: An high level representation of the proposed solution.
• Figure 3: The architecture of the neural network used for keyword spotting.
• Figure 4: The architecture of the neural network used for extracting the d-vectors.
• Figure 5: The adaptation phase of the proposed adaptive speaker verification model.