Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Audio Decoding by Inverse Problem Solving

Pedro J. Villasana T., Lars Villemoes, Janusz Klejsa, Per Hedelin

TL;DR

The noisy mean model, underlying the proposed derivation of conditioning, enables a significant reduction of gradient evaluations for diffusion posterior sampling, compared to methods based on Tweedie’s mean, which improves the objective performance.

Abstract

We consider audio decoding as an inverse problem and solve it through diffusion posterior sampling. Explicit conditioning functions are developed for input signal measurements provided by an example of a transform domain perceptual audio codec. Viability is demonstrated by evaluating arbitrary pairings of a set of bitrates and task-agnostic prior models. For instance, we observe significant improvements on piano while maintaining speech performance when a speech model is replaced by a joint model trained on both speech and piano. With a more general music model, improved decoding compared to legacy methods is obtained for a broad range of content types and bitrates. The noisy mean model, underlying the proposed derivation of conditioning, enables a significant reduction of gradient evaluations for diffusion posterior sampling, compared to methods based on Tweedie's mean. Combining Tweedie's mean with our conditioning functions improves the objective performance. An audio demo is available at https://dpscodec-demo.github.io/.

Audio Decoding by Inverse Problem Solving

TL;DR

The noisy mean model, underlying the proposed derivation of conditioning, enables a significant reduction of gradient evaluations for diffusion posterior sampling, compared to methods based on Tweedie’s mean, which improves the objective performance.

Abstract

We consider audio decoding as an inverse problem and solve it through diffusion posterior sampling. Explicit conditioning functions are developed for input signal measurements provided by an example of a transform domain perceptual audio codec. Viability is demonstrated by evaluating arbitrary pairings of a set of bitrates and task-agnostic prior models. For instance, we observe significant improvements on piano while maintaining speech performance when a speech model is replaced by a joint model trained on both speech and piano. With a more general music model, improved decoding compared to legacy methods is obtained for a broad range of content types and bitrates. The noisy mean model, underlying the proposed derivation of conditioning, enables a significant reduction of gradient evaluations for diffusion posterior sampling, compared to methods based on Tweedie's mean. Combining Tweedie's mean with our conditioning functions improves the objective performance. An audio demo is available at https://dpscodec-demo.github.io/.
Paper Structure (18 sections, 20 equations, 4 figures, 3 tables)

This paper contains 18 sections, 20 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Perceptual Audio Codec
  3. Encoder
  4. Legacy Decoder
  5. Audio Decoding as Inverse Problem Solving
  6. Langevin Sampling
  7. Score Matching
  8. Conditional Scores
  9. Unitary Transforms
  10. Band-wise Conditional Scores
  11. Envelope Quantization Scores
  12. Sample Quantization Scores
  13. Conditional Scores with Tweedie's Mean
  14. Performance Analysis
  15. Experimental Setup
  16. ...and 3 more sections

Figures (4)

  • Figure 1: A perceptual audio codec.
  • Figure 2: Spectrograms: (left) Input signal; (center-left) Deterministic reconstruction; (center-right) Legacy-decoded signal; (right) Diffusion-decoded signal.
  • Figure 3: Results of a MUSHRA-like listening test at 16 kb/s (3 items per category, 10 listeners, Student-t 95% confidence intervals).
  • Figure 4: Results of a MUSHRA-like listening test at 16 kb/s on the Critical test set (10 listeners, Student-t 95% confidence intervals).