Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Domain Adaptation for Learned Image Compression with Supervised Adapters

Alberto Presta, Gabriele Spadaro, Enzo Tartaglione, Attilio Fiandrotti, Marco Grangetto

TL;DR

This work tackles cross-domain robustness in Learned Image Compression (LIC) by introducing decoder-side, domain-specific adapters and a gate network that blends their outputs. It deploys $K+1$ residual adapters (one per domain: $K$ targets plus the source) and a gate producing $\mathbf{v} \in [0,1]^{K+1}$ to weight adapter contributions, while keeping the pre-trained encoder/decoder fixed. Adapters and gate are trained jointly with a loss $\mathcal{L} = \gamma \cdot \mathrm{MSE}(\mathbf{x}, \hat{\mathbf{x}}) + \mathrm{CE}(d, \mathbf{v})$, avoiding encoder refinement and encoder-side rate changes. Results show consistent rate–distortion gains on target domains with no forgetting of the source, and the gate demonstrates reasonable generalization to unseen domains, indicating scalable cross-domain LIC with modest overhead (~3.8M adapter params and ~184k gate params). The approach offers practical benefits for deploying LIC across diverse content without retraining the entire model or transmitting per-image adapters.

Abstract

In Learned Image Compression (LIC), a model is trained at encoding and decoding images sampled from a source domain, often outperforming traditional codecs on natural images; yet its performance may be far from optimal on images sampled from different domains. In this work, we tackle the problem of adapting a pre-trained model to multiple target domains by plugging into the decoder an adapter module for each of them, including the source one. Each adapter improves the decoder performance on a specific domain, without the model forgetting about the images seen at training time. A gate network computes the weights to optimally blend the contributions from the adapters when the bitstream is decoded. We experimentally validate our method over two state-of-the-art pre-trained models, observing improved rate-distortion efficiency on the target domains without penalties on the source domain. Furthermore, the gate's ability to find similarities with the learned target domains enables better encoding efficiency also for images outside them.

Domain Adaptation for Learned Image Compression with Supervised Adapters

TL;DR

This work tackles cross-domain robustness in Learned Image Compression (LIC) by introducing decoder-side, domain-specific adapters and a gate network that blends their outputs. It deploys residual adapters (one per domain: targets plus the source) and a gate producing to weight adapter contributions, while keeping the pre-trained encoder/decoder fixed. Adapters and gate are trained jointly with a loss , avoiding encoder refinement and encoder-side rate changes. Results show consistent rate–distortion gains on target domains with no forgetting of the source, and the gate demonstrates reasonable generalization to unseen domains, indicating scalable cross-domain LIC with modest overhead (~3.8M adapter params and ~184k gate params). The approach offers practical benefits for deploying LIC across diverse content without retraining the entire model or transmitting per-image adapters.

Abstract

In Learned Image Compression (LIC), a model is trained at encoding and decoding images sampled from a source domain, often outperforming traditional codecs on natural images; yet its performance may be far from optimal on images sampled from different domains. In this work, we tackle the problem of adapting a pre-trained model to multiple target domains by plugging into the decoder an adapter module for each of them, including the source one. Each adapter improves the decoder performance on a specific domain, without the model forgetting about the images seen at training time. A gate network computes the weights to optimally blend the contributions from the adapters when the bitstream is decoded. We experimentally validate our method over two state-of-the-art pre-trained models, observing improved rate-distortion efficiency on the target domains without penalties on the source domain. Furthermore, the gate's ability to find similarities with the learned target domains enables better encoding efficiency also for images outside them.
Paper Structure (15 sections, 3 equations, 6 figures, 2 tables)

This paper contains 15 sections, 3 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background and related works
  3. Learned image compression
  4. Task-domain adaptation for learned image compression
  5. Proposed method and architecture
  6. Domain adaptation at the decoder
  7. Gate Network
  8. Adapters blending policy
  9. Training the adapters and the gate
  10. Experiments and Results
  11. Experimental setup
  12. Rate-Distortion Performance
  13. Comparison with other blending policies
  14. Unseen domains
  15. Conclusions and future works

Figures (6)

  • Figure 1: (a) Architecture of the WAM adapter $Ad^{k}_{0}$. (b) Architecture of the deconvolutional adapters $Ad^{k}_{\{1,2\}}$. (c) Overall decoder. (d) Gate network $\varphi$.
  • Figure 2: (a) Adapters blending policy based on weighted sum. Index 0 refers to the source domain adapter. (b) Blueprint of a LIC model with the gate.
  • Figure 3: Rate-PSNR plots for our Adapter-based method vs Reference for Zou et al. (blue) and Cheng et al. (orange), considering Sketch (a) and Comic (b).
  • Figure 4: BD-Rate/PSNR for different domains.
  • Figure 5: Avg. probabilities predicted by $\varphi$.
  • ...and 1 more figures