High-Fidelity Generative Audio Compression at 0.275kbps

TL;DR

The paper tackles ultra-low bitrate general audio compression by shifting from waveform fidelity to semantic transmission using Generative Audio Compression (GAC) within the AI Flow framework. It grounds the method in the Law of Information Capacity (IC-1) and formulates a two-stage pipeline: Stage 1 compresses semantic content into a discrete latent Z via an Information Bottleneck objective, and Stage 2 reconstructs high-fidelity audio from Z with a large generative decoder (rectified-flow based) to compensate limited bandwidth. A 1.8B-parameter model achieves 32kHz audio at 0.275 kbps, with intelligibility maintained down to 0.175 kbps, outperforming state-of-the-art neural codecs across speech, general sounds, and music. The results demonstrate that increasing receiver computation can offset extreme channel bottlenecks, enabling perceptually faithful and semantically coherent audio at drastically reduced bitrates, with strong implications for low-bandwidth communication and generative audio applications.

Abstract

High-fidelity general audio compression at ultra-low bitrates is crucial for applications ranging from low-bandwidth communication to generative audio-language modeling. Traditional audio compression methods and contemporary neural codecs are fundamentally designed for waveform reconstruction. As a result, when operating at ultra-low bitrates, these methods degrade rapidly and often fail to preserve essential information, leading to severe acoustic artifacts and pronounced semantic distortion. To overcome these limitations, we introduce Generative Audio Compression (GAC), a novel paradigm shift from signal fidelity to task-oriented effectiveness. Implemented within the AI Flow framework, GAC is theoretically grounded in the Law of Information Capacity. These foundations posit that abundant computational power can be leveraged at the receiver to offset extreme communication bottlenecks--exemplifying the More Computation, Less Bandwidth philosophy. By integrating semantic understanding at the transmitter with scalable generative synthesis at the receiver, GAC offloads the information burden to powerful model priors. Our 1.8B-parameter model achieves high-fidelity reconstruction of 32kHz general audio at an unprecedented bitrate of 0.275kbps. Even at 0.175kbps, it still preserves a strong intelligible audio transmission capability, which represents an about 3000x compression ratio, significantly outperforming current state-of-the-art neural codecs in maintaining both perceptual quality and semantic consistency.

Figures (3)

• Figure 1: Rate-quality comparison of various audio compression methods. The y-axis denotes the average normalized objective metrics across speech, sound, and music domains. Marker size reflects the number of parameters. Our method (1.8B) consistently outperforms baselines, achieving high perceptual quality under 1kbps bitrates.
• Figure 2: Illustration of the generative audio compression framework.
• Figure 3: Illustration of the model performance varying with the bandwidth of different-sized models in the speech, sound, and music domains.