Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Test-time adaptation for image compression with distribution regularization

Kecheng Chen, Pingping Zhang, Tiexin Qin, Shiqi Wang, Hong Yan, Haoliang Li

TL;DR

This work aims to develop an advanced latent refinement method by extending the effective hybrid latent refinement (HLR) method, and introduces a simple Bayesian approximation-endowed distribution regularization to encourage learning a better joint probability approximation in a plug-and-play manner.

Abstract

Current test- or compression-time adaptation image compression (TTA-IC) approaches, which leverage both latent and decoder refinements as a two-step adaptation scheme, have potentially enhanced the rate-distortion (R-D) performance of learned image compression models on cross-domain compression tasks, \textit{e.g.,} from natural to screen content images. However, compared with the emergence of various decoder refinement variants, the latent refinement, as an inseparable ingredient, is barely tailored to cross-domain scenarios. To this end, we aim to develop an advanced latent refinement method by extending the effective hybrid latent refinement (HLR) method, which is designed for \textit{in-domain} inference improvement but shows noticeable degradation of the rate cost in \textit{cross-domain} tasks. Specifically, we first provide theoretical analyses, in a cue of marginalization approximation from in- to cross-domain scenarios, to uncover that the vanilla HLR suffers from an underlying mismatch between refined Gaussian conditional and hyperprior distributions, leading to deteriorated joint probability approximation of marginal distribution with increased rate consumption. To remedy this issue, we introduce a simple Bayesian approximation-endowed \textit{distribution regularization} to encourage learning a better joint probability approximation in a plug-and-play manner. Extensive experiments on six in- and cross-domain datasets demonstrate that our proposed method not only improves the R-D performance compared with other latent refinement counterparts, but also can be flexibly integrated into existing TTA-IC methods with incremental benefits.

Test-time adaptation for image compression with distribution regularization

TL;DR

This work aims to develop an advanced latent refinement method by extending the effective hybrid latent refinement (HLR) method, and introduces a simple Bayesian approximation-endowed distribution regularization to encourage learning a better joint probability approximation in a plug-and-play manner.

Abstract

Current test- or compression-time adaptation image compression (TTA-IC) approaches, which leverage both latent and decoder refinements as a two-step adaptation scheme, have potentially enhanced the rate-distortion (R-D) performance of learned image compression models on cross-domain compression tasks, \textit{e.g.,} from natural to screen content images. However, compared with the emergence of various decoder refinement variants, the latent refinement, as an inseparable ingredient, is barely tailored to cross-domain scenarios. To this end, we aim to develop an advanced latent refinement method by extending the effective hybrid latent refinement (HLR) method, which is designed for \textit{in-domain} inference improvement but shows noticeable degradation of the rate cost in \textit{cross-domain} tasks. Specifically, we first provide theoretical analyses, in a cue of marginalization approximation from in- to cross-domain scenarios, to uncover that the vanilla HLR suffers from an underlying mismatch between refined Gaussian conditional and hyperprior distributions, leading to deteriorated joint probability approximation of marginal distribution with increased rate consumption. To remedy this issue, we introduce a simple Bayesian approximation-endowed \textit{distribution regularization} to encourage learning a better joint probability approximation in a plug-and-play manner. Extensive experiments on six in- and cross-domain datasets demonstrate that our proposed method not only improves the R-D performance compared with other latent refinement counterparts, but also can be flexibly integrated into existing TTA-IC methods with incremental benefits.

Paper Structure

This paper contains 14 sections, 4 theorems, 20 equations, 23 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Methodology
  4. In-domain Marginalization Approximation
  5. Generalize to cross-domain marginalization approximation
  6. Distribution regularization via Bayesian approximation
  7. Experiments
  8. Comparison with Latent Refinement Methods
  9. Comparison with SOTA TTA-IC -- Plugging into Distribution Regularization
  10. In-depth Analyses of Our Proposed Method
  11. Conclusion
  12. Details of adopted datasets
  13. Comparison with SOTA TTA-IC Methods
  14. Connection with bit-back coding

Key Result

Lemma 1

Let $p(y|z)$ and $p(z)$ be accessed, a joint probability over $y$ and $z$ can be constructed to approximate the true marginal probability over $y$, where $x$, $g_{a}(\cdot)$, and $h_{a}(\cdot)$ denote a raw image, an analysis transform function of the entropy model, and an analysis transform function of the hyperprior model.

Figures (23)

  • Figure 1: Comparison of various latent refinement methods in R-D performance under cross-domain tasks (testing on SIQAD screen content dataset). The Cheng20 cheng2020learned (quality$=0$, pre-trained on natural images) model is used.
  • Figure 2: BLR
  • Figure 3: HLR
  • Figure 4: Ours
  • Figure 6:
  • ...and 18 more figures

Theorems & Definitions (7)

  • Lemma 1: balle2018variational
  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • Corollary 1
  • proof