Q2D2: A Geometry-Aware Audio Codec Leveraging Two-Dimensional Quantization

TL;DR

Q2D2 introduces a geometry-aware two-dimensional quantizer that groups latent channels into pairs and quantizes them on fixed grids (hexagonal, rhombic, rectangular). This yields an implicit codebook defined by the product of pairwise grids, enabling high codebook utilization and very low token rates while preserving state-of-the-art reconstruction quality. Extensive experiments across LibriTTS, LibriSpeech, and LJSpeech show competitive or superior performance at 1–6.9 kbps with far fewer tokens than baselines, with ablations highlighting the benefits of rhombic tilings and moderate dimensionality. The work suggests that structured 2D quantization robustly captures inter-feature correlations and offers a promising alternative to conventional scalar or vector quantization in neural audio codecs, with future extensions to 3D grids and broader audio domains.

Abstract

Recent neural audio codecs have achieved impressive reconstruction quality, typically relying on quantization methods such as Residual Vector Quantization (RVQ), Vector Quantization (VQ) and Finite Scalar Quantization (FSQ). However, these quantization techniques limit the geometric structure of the latent space, make it harder to capture correlations between features leading to inefficiency in representation learning, codebook utilization and token rate. In this paper we introduce Two Dimensional Quantization (Q2D2), a quantization scheme in which feature pairs are projected onto structured 2D grids such as hexagonal, rhombic, or rectangular tiling and quantized to the nearest grid values, yielding an implicit codebook defined by the product of grid levels, with codebook sizes comparable to conventional methods. Despite its simple geometric formulation, Q2D2 improves audio compression efficiency, with low token rates and high codebook utilization while maintaining state of the art reconstruction quality. Specifically, Q2D2 achieves competitive to superior performance in various objective and subjective reconstruction metrics, across extensive experiments in speech domain compared to state of the art models. Comprehensive ablation studies further confirm the effectiveness of our design choices.

Figures (4)

• Figure 1: Visualization of quantization grids used in Q2D2: Hexagonal Grid (a): a hexagonal tiling with 9 quantization levels in $x$ and $y$ axis. Rectangle Grid (b): a rectangle tiling with 7 quantization levels in $x$ and $y$ axis. Rhombic Grid (c): a rhombic tiling with 7 quantization levels in $x$ axis, and 6 levels yielding to $11$ quantization levels in $y$ axis.
• Figure 2: Q2D2 (a): The final encoder layer is projected to $d$ selected latent feature dimensions. Each projected dimension is first bounded between $[-{l_i}/2, {l_i}/2]$, where $l_i$ is the number of levels selected per dimension. Q2D2 then groups the dimensions into pairs (in example, 6 dimensions are reshaped into 3 pairs), and jointly quantizes each pair onto a structured 2D grid and finding the nearest point on the grid. FSQ (b): The final encoder layer is projected to $d$ dimensions (example with $d=3$). Each projected dimension $z$ is bounded to $l$ discrete values (here $l=3$), and then rounded to the nearest integer, producing the quantized vector $\hat{{\bm{z}}}$, the nearest point in the hypercube. VQ (c): The final encoder layer is projected to $d$ dimensions (example shown with $d=5$, as $d$ is typically larger in VQ). The latent vector ${\bm{z}}$ is replaced by the closest vector from the codebook $\hat{{\bm{z}}} \in {\mathbb{C}}$ via nearest-neighbor lookup.
• Figure 3: Comparison between different acoustic codec models. The y-axis UTMOS reflects reconstruction quality (UTMOS highly correlates with human evaluations), the x-axis kbps represents audio compression levels. The size of circles represents the number of discrete tokens per second.
• Figure 4: Mutual information between the two coordinates of each 2D pair before and after quantization. Pre-quantization MI is near zero, indicating independent components, while post-quantization MI increases significantly, showing that the 2D grid introduces structured dependencies.