Online Meta Adaptation for Variable-Rate Learned Image Compression

TL;DR

This paper tackles two challenges in learned image compression: achieving true variable-rate performance without training separate models, and closing the gap between training-time soft quantization and test-time hard quantization. It introduces Online Meta-Learning (OML) within a CVAE framework, updating conditional meta-parameters via SGD on a per-image basis to steer reconstruction quality according to the current rate-distortion target $\lambda$, without changing encoder/decoder parameters. By applying online adaptation to the latent-domain via conditional feature modulation, OML bridges the training-test mismatch and enables fast per-image RD adjustments with negligible transmission overhead. Empirical results on JPEG-AI MMSP 2020 show consistent improvements over state-of-the-art LIC baselines, especially at low bitrates, with modest computation overhead and applicability to multiple LIC architectures.

Abstract

This work addresses two major issues of end-to-end learned image compression (LIC) based on deep neural networks: variable-rate learning where separate networks are required to generate compressed images with varying qualities, and the train-test mismatch between differentiable approximate quantization and true hard quantization. We introduce an online meta-learning (OML) setting for LIC, which combines ideas from meta learning and online learning in the conditional variational auto-encoder (CVAE) framework. By treating the conditional variables as meta parameters and treating the generated conditional features as meta priors, the desired reconstruction can be controlled by the meta parameters to accommodate compression with variable qualities. The online learning framework is used to update the meta parameters so that the conditional reconstruction is adaptively tuned for the current image. Through the OML mechanism, the meta parameters can be effectively updated through SGD. The conditional reconstruction is directly based on the quantized latent representation in the decoder network, and therefore helps to bridge the gap between the training estimation and true quantized latent distribution. Experiments demonstrate that our OML approach can be flexibly applied to different state-of-the-art LIC methods to achieve additional performance improvements with little computation and transmission overhead.

Figures (7)

• Figure 1: Conditional feature modulation
• Figure 2: Architecture of the proposed variable-rate meta-LIC method. The conditional feature modulators are applied to the decoder to perform channel-wise modulation over intermediate decoding features after the decoding blocks.
• Figure 3: Online meta-update of conditional tradeoff parameters through SGD.
• Figure 4: Performance improvements of OML over different base models. The MSSSIM is magnified as: $-10\log_{10}(1-\text{MSSSIM})$
• Figure 5: Performance vs. time for different iterations.