OASI: Objective-Aware Surrogate Initialization for Multi-Objective Bayesian Optimization in TinyML Keyword Spotting

TL;DR

The paper tackles the problem of balancing accuracy and resource usage in TinyML keyword spotting by casting hyperparameter optimization as a constrained multi-objective problem. It introduces Objective-Aware Surrogate Initialization (OASI), a MOSA-based seeding method that creates a Pareto-focused initialization for MOBO, improving early surrogate quality and convergence. Through extensive experiments on Google Speech Commands v2 with a DS-CNN backbone, OASI achieves the best Pareto-front proxies, attaining a hypervolume of 0.0627 and a generational distance of 0.0, with top models delivering >90% accuracy at around 0.10 MB. The results demonstrate meaningful practical impact for deploying accurate, privacy-preserving KWS on ultra-low-power devices and suggest broader applicability to other constrained MOBO scenarios in TinyML.

Abstract

Voice assistants utilize Keyword Spotting (KWS) to enable efficient, privacy-friendly activation. However, realizing accurate KWS models on ultra-low-power TinyML devices (often with less than $<2$ MB of flash memory) necessitates a delicate balance between accuracy with strict resource constraints. Multi-objective Bayesian Optimization (MOBO) is an ideal candidate for managing such a trade-off but is highly initialization-dependent, especially under the budgeted black-box setting. Existing methods typically fall back to naive, ad-hoc sampling routines (e.g., Latin Hypercube Sampling (LHS), Sobol sequences, or Random search) that are adapted to neither the Pareto front nor undergo rigorous statistical comparison. To address this, we propose Objective-Aware Surrogate Initialization (OASI), a novel initialization strategy that leverages Multi-Objective Simulated Annealing (MOSA) to generate a seed Pareto set of high-performing and diverse configurations that explicitly balance accuracy and model size. Evaluated in a TinyML KWS setting, OASI outperforms LHS, Sobol, and Random initialization, achieving the highest hypervolume (0.0627) and the lowest generational distance (0.0) across multiple runs, with only a modest increase in computation time (1934 s vs. $\sim$1500 s). A non-parametric statistical analysis using the Kruskal-Wallis test ($H = 5.40$, $p = 0.144$, $η^2 = 0.0007$) and Dunn's post-hoc test confirms OASI's superior consistency despite the non-significant overall difference with respect to the $α=0.05$ threshold.

Figures (6)

• Figure 1: Block diagram of the bi-level multi-objective optimization process: sampling, training, evaluation, archiving, and Pareto front extraction.
• Figure 2: MOBO framework with proposed OASI framework: MOSA-based initialization with GP-surrogate MOBO for Pareto-optimal KWS models
• Figure 3: A general architecture of the DSCNN highlighting the key hyperparameters used in this work.
• Figure 4: Effect of initialization strategies on MOBO.
• Figure 5: Comparison of initialization strategies in terms of (a) best validation accuracy progression and (b) hypervolume progression over time.