Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Categorical Unsupervised Variational Acoustic Clustering

Luan Vinícius Fiorio, Ivana Nikoloska, Ronald M. Aarts

TL;DR

The paper addresses unsupervised clustering of audio in time-frequency representations where data strongly overlap, such as urban scenes. It extends UVAC by introducing a categorical latent variable implemented via a differentiable Gumbel-Softmax with a tunable temperature $\tau$ to control cluster sharpness. A variational inference framework yields an ELBO with reconstruction and KL terms, enabling end-to-end training. Empirical results on AudioMNIST, TAU2019, and UrbanSound8K show that Cat. UVAC outperforms Gaussian-based UVAC and K-means, particularly in overlapped data, with clustering metrics improving as the cluster count is reduced. This method offers a practical unsupervised clustering approach for audio with potential applications in hearing aids and related processing pipelines, leveraging the discrete latent structure to achieve sharp, meaningful clusters.

Abstract

We propose a categorical approach for unsupervised variational acoustic clustering of audio data in the time-frequency domain. The consideration of a categorical distribution enforces sharper clustering even when data points strongly overlap in time and frequency, which is the case for most datasets of urban acoustic scenes. To this end, we use a Gumbel-Softmax distribution as a soft approximation to the categorical distribution, allowing for training via backpropagation. In this settings, the softmax temperature serves as the main mechanism to tune clustering performance. The results show that the proposed model can obtain impressive clustering performance for all considered datasets, even when data points strongly overlap in time and frequency.

Categorical Unsupervised Variational Acoustic Clustering

TL;DR

The paper addresses unsupervised clustering of audio in time-frequency representations where data strongly overlap, such as urban scenes. It extends UVAC by introducing a categorical latent variable implemented via a differentiable Gumbel-Softmax with a tunable temperature to control cluster sharpness. A variational inference framework yields an ELBO with reconstruction and KL terms, enabling end-to-end training. Empirical results on AudioMNIST, TAU2019, and UrbanSound8K show that Cat. UVAC outperforms Gaussian-based UVAC and K-means, particularly in overlapped data, with clustering metrics improving as the cluster count is reduced. This method offers a practical unsupervised clustering approach for audio with potential applications in hearing aids and related processing pipelines, leveraging the discrete latent structure to achieve sharp, meaningful clusters.

Abstract

We propose a categorical approach for unsupervised variational acoustic clustering of audio data in the time-frequency domain. The consideration of a categorical distribution enforces sharper clustering even when data points strongly overlap in time and frequency, which is the case for most datasets of urban acoustic scenes. To this end, we use a Gumbel-Softmax distribution as a soft approximation to the categorical distribution, allowing for training via backpropagation. In this settings, the softmax temperature serves as the main mechanism to tune clustering performance. The results show that the proposed model can obtain impressive clustering performance for all considered datasets, even when data points strongly overlap in time and frequency.

Paper Structure

This paper contains 14 sections, 9 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Variational inference
  3. Sampling from a categorical distribution
  4. Architecture
  5. Experimental evaluation
  6. Tasks
  7. Spoken digit recognition
  8. Urban acoustic scene classification
  9. Metrics
  10. Baselines
  11. Hyperparameters
  12. Results
  13. Conclusion
  14. Acknowledgment

Figures (2)

  • Figure 1: Diagram of the categorical UVAC model. $q_\upsilon(y|\mathbf{x})$ is represented by a NN $h_\upsilon(\mathbf{x})=[\boldsymbol{\pi}_k]$ and $\tau$ is the Gumbel-Softmax temperature. $f_\phi(y,\mathbf{x})=[\boldsymbol{\mu}_\phi, \boldsymbol{\Sigma}_\phi]$ is a NN equivalent to $q_\phi(\mathbf{z}|y,\mathbf{x})$. The decoder $p_\theta(\mathbf{x}|y,\mathbf{z})$ is a NN $g_\theta(y,\mathbf{z}) = [\hat{\mathbf{x}}]$.
  • Figure 2: Gumbel-Softmax distribution example plot for 10 classes with different values of $\tau$.