Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

3R-INN: How to be climate friendly while consuming/delivering videos?

Zoubida Ameur, Claire-Hélène Demarty, Daniel Menard, Olivier Le Meur

TL;DR

3R-INN is proposed, a single light invertible network that does three tasks at once: given a high-resolution grainy image, it Rescales it to a lower resolution, Removes film grain and reduces its power consumption when displayed, and outperforms state-of-the-art film grain synthesis and energy-aware methods.

Abstract

The consumption of a video requires a considerable amount of energy during the various stages of its life-cycle. With a billion hours of video consumed daily, this contributes significantly to the greenhouse gas emission. Therefore, reducing the end-to-end carbon footprint of the video chain, while preserving the quality of experience at the user side, is of high importance. To contribute in an impactful manner, we propose 3R-INN, a single light invertible network that does three tasks at once: given a high-resolution grainy image, it Rescales it to a lower resolution, Removes film grain and Reduces its power consumption when displayed. Providing such a minimum viable quality content contributes to reducing the energy consumption during encoding, transmission, decoding and display. 3R-INN also offers the possibility to restore either the high-resolution grainy original image or a grain-free version, thanks to its invertibility and the disentanglement of the high frequency, and without transmitting auxiliary data. Experiments show that, while enabling significant energy savings for encoding (78%), decoding (77%) and rendering (5% to 20%), 3R-INN outperforms state-of-the-art film grain synthesis and energy-aware methods and achieves state-of-the-art performance on the rescaling task on different test-sets.

3R-INN: How to be climate friendly while consuming/delivering videos?

TL;DR

3R-INN is proposed, a single light invertible network that does three tasks at once: given a high-resolution grainy image, it Rescales it to a lower resolution, Removes film grain and reduces its power consumption when displayed, and outperforms state-of-the-art film grain synthesis and energy-aware methods.

Abstract

The consumption of a video requires a considerable amount of energy during the various stages of its life-cycle. With a billion hours of video consumed daily, this contributes significantly to the greenhouse gas emission. Therefore, reducing the end-to-end carbon footprint of the video chain, while preserving the quality of experience at the user side, is of high importance. To contribute in an impactful manner, we propose 3R-INN, a single light invertible network that does three tasks at once: given a high-resolution grainy image, it Rescales it to a lower resolution, Removes film grain and Reduces its power consumption when displayed. Providing such a minimum viable quality content contributes to reducing the energy consumption during encoding, transmission, decoding and display. 3R-INN also offers the possibility to restore either the high-resolution grainy original image or a grain-free version, thanks to its invertibility and the disentanglement of the high frequency, and without transmitting auxiliary data. Experiments show that, while enabling significant energy savings for encoding (78%), decoding (77%) and rendering (5% to 20%), 3R-INN outperforms state-of-the-art film grain synthesis and energy-aware methods and achieves state-of-the-art performance on the rescaling task on different test-sets.
Paper Structure (17 sections, 12 equations, 8 figures, 5 tables)

This paper contains 17 sections, 12 equations, 8 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Proposed approach
  4. Haar transform
  5. Invertible block
  6. Conditioned latent encoding block
  7. Training objectives
  8. Rescaling and film grain removal/synthesis tasks
  9. Energy-aware task
  10. Experiments
  11. Training details
  12. Evaluation of downscaled LR images
  13. Evaluation of generated HR clean images
  14. Evaluation of reconstructed HR grainy images
  15. Ablation study
  16. ...and 2 more sections

Figures (8)

  • Figure 1: 3R-INN: End-to-end energy-aware video distribution chain by Removing grain, Rescaling and Reducing display energy.
  • Figure 2: Comparison between a bicubic downscaling, IRN and the generated clean lr image $\Tilde{I}_{LR|R=0}$.
  • Figure 3: Comparison of generated energy-aware images with the state-of-the-art, for $R \in \{5\%, 20\%, 40\%\}$ from first to third lines. Achieved rates computed by the power model in Demarty2023 are provided.
  • Figure 4: Qualitative evaluation of hr synthesized grainy images for different methods, with LPIPS values.
  • Figure 5: SSIM scores as function of the target power reduction, for the different energy-aware methods.
  • ...and 3 more figures