Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

M-Vec: Matryoshka Speaker Embeddings with Flexible Dimensions

Shuai Wang, Pengcheng Zhu, Haizhou Li

TL;DR

This work tackles the inefficiency of fixed-dimensional speaker embeddings by introducing Matryoshka Representation Learning (MRL), which enables nested, variable-dimension embeddings that can be extracted at inference without retraining. By jointly training multiple sub-dimensions under an AAM-Softmax framework, the method maintains discriminability even at very low dimensions (e.g., $8$ or $16$), as demonstrated on VoxCeleb data. The key contributions are the MRL loss that aggregates across dimensions, and the practical demonstration that significant storage and retrieval-time reductions are achievable with minimal performance loss, making large-scale speaker databases more scalable. The approach is compatible with existing encoders and can be extended to other tasks requiring flexible embedding dimensionality, offering tangible benefits for deployment and retrieval efficiency.

Abstract

Fixed-dimensional speaker embeddings have become the dominant approach in speaker modeling, typically spanning hundreds to thousands of dimensions. These dimensions are hyperparameters that are not specifically picked, nor are they hierarchically ordered in terms of importance. In large-scale speaker representation databases, reducing the dimensionality of embeddings can significantly lower storage and computational costs. However, directly training low-dimensional representations often yields suboptimal performance. In this paper, we introduce the Matryoshka speaker embedding, a method that allows dynamic extraction of sub-dimensions from the embedding while maintaining performance. Our approach is validated on the VoxCeleb dataset, demonstrating that it can achieve extremely low-dimensional embeddings, such as 8 dimensions, while preserving high speaker verification performance.

M-Vec: Matryoshka Speaker Embeddings with Flexible Dimensions

TL;DR

This work tackles the inefficiency of fixed-dimensional speaker embeddings by introducing Matryoshka Representation Learning (MRL), which enables nested, variable-dimension embeddings that can be extracted at inference without retraining. By jointly training multiple sub-dimensions under an AAM-Softmax framework, the method maintains discriminability even at very low dimensions (e.g., or ), as demonstrated on VoxCeleb data. The key contributions are the MRL loss that aggregates across dimensions, and the practical demonstration that significant storage and retrieval-time reductions are achievable with minimal performance loss, making large-scale speaker databases more scalable. The approach is compatible with existing encoders and can be extended to other tasks requiring flexible embedding dimensionality, offering tangible benefits for deployment and retrieval efficiency.

Abstract

Fixed-dimensional speaker embeddings have become the dominant approach in speaker modeling, typically spanning hundreds to thousands of dimensions. These dimensions are hyperparameters that are not specifically picked, nor are they hierarchically ordered in terms of importance. In large-scale speaker representation databases, reducing the dimensionality of embeddings can significantly lower storage and computational costs. However, directly training low-dimensional representations often yields suboptimal performance. In this paper, we introduce the Matryoshka speaker embedding, a method that allows dynamic extraction of sub-dimensions from the embedding while maintaining performance. Our approach is validated on the VoxCeleb dataset, demonstrating that it can achieve extremely low-dimensional embeddings, such as 8 dimensions, while preserving high speaker verification performance.
Paper Structure (17 sections, 3 equations, 2 figures, 2 tables)

This paper contains 17 sections, 3 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Speaker Modeling for Human-Computer Interaction
  3. Background on speaker embedding learning
  4. Extremely Low-Dimensional Embeddings
  5. Contributions
  6. Matryoshka Embedding Learning
  7. Matryoshka Representation
  8. MRL for Speaker Embedding Learning
  9. Experiments
  10. Dataset
  11. Experimental Setups
  12. Results and Analysis
  13. Comparison of Embeddings with Different Dimensions
  14. Extremely Low Dimensional Embeddings
  15. Analysis on the Storage and Retivial Time
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Matryoshca Speaker Embedding Learning, we use 3 sub-dimensional embeddings as a illustration
  • Figure 2: Performance comparison of different systems using different dimensions