Clustering of Acoustic Environments with Variational Autoencoders for Hearing Devices

TL;DR

This work tackles unsupervised clustering of acoustic environments for hearing devices, addressing label scarcity and strong time-frequency overlap in audio. It advances a variational autoencoder framework that supports both continuous (VIB-GMM) and continuous-categorical (M2-AC) latent spaces, incorporating a Gumbel-Softmax reparameterization and a time-context windowing scheme to operate under hardware constraints. Empirical results show that a continuous-categorical M2-AC architecture delivers robust clustering for urban soundscapes and spoken digits, with windowed variants offering further practical benefits. The findings indicate that categorical latent representations are better suited for clustering highly overlapping audio, enabling potential unsupervised adaptation of hearing-device processing to different soundscapes.

Abstract

Traditional acoustic environment classification relies on: i) classical signal processing algorithms, which are unable to extract meaningful representations of high-dimensional data; or on ii) supervised learning, limited by the availability of labels. Knowing that human-imposed labels do not always reflect the true structure of acoustic scenes, we explore the potential of (unsupervised) clustering of acoustic environments using variational autoencoders (VAEs). We employ a VAE model for categorical latent clustering with a Gumbel-Softmax reparameterization which can operate with a time-context windowing scheme for lower memory requirements, tailored for real-world hearing device scenarios. Additionally, general adaptations on VAE architectures for audio clustering are also proposed. The approaches are validated through the clustering of spoken digits, a simpler task where labels are meaningful, and urban soundscapes, where the recordings present strong overlap in time and frequency. While all variational methods succeeded when clustering spoken digits, only the proposed model achieved effective clustering performance on urban acoustic scenes, given its categorical nature.

Figures (8)

• Figure 1: Inference/generative model for clustering with a continuous latent variable $\mathbf{z}$.
• Figure 2: Inference/generative model for clustering with a categorical ($y$) and a continuous ($\mathbf{z}$) latent variable.
• Figure 3: Gumbel-Softmax distribution example plot for 10 classes with different values of $\tau$.
• Figure 4: Example of sliding window clustering inference for $w=3$, hop of 1 sample, with $N=5$ samples. $\mathbf{x}(n)$ is the $n$-th time bin of an input sample $\mathbf{x}^{(i)}$, $\mathbf{x}_{j}$ is the input window $j$ with estimated logits $\boldsymbol{\pi}_{j}$, $\bar{\boldsymbol{\pi}}$ are the average logits, $\mathrm{GS}$ is the Gumbel-Softmax function, and $y$ are the cluster probabilities.
• Figure 5: Diagram of the neural network-based VAE clustering model with a (continuous) Gaussian mixture model prior. $q_\phi(\mathbf{z}|\mathbf{x})$ is represented by a NN $f_\phi(\mathbf{x}) = [\boldsymbol{\mu}_\phi, \boldsymbol{\Sigma}_\phi]$. The decoder $p_\theta(\mathbf{x}|\mathbf{z})$ is a NN $g_\theta(\mathbf{z}) = [\hat{\mathbf{x}}]$.