Block Modulating Video Compression: An Ultra Low Complexity Image Compression Encoder for Resource Limited Platforms

TL;DR

An ultra low-cost image encoder, named Block Modulating Video Compression (BMVC) with an encoding complexity of O(1) is proposed to be implemented on mobile platforms with low consumption of power and computation resources.

Abstract

We consider the image and video compression on resource limited platforms. An ultra low-cost image encoder, named Block Modulating Video Compression (BMVC) with an encoding complexity ${\cal O}(1)$ is proposed to be implemented on mobile platforms with low consumption of power and computation resources. We also develop two types of BMVC decoders, implemented by deep neural networks. The first BMVC decoder is based on the Plug-and-Play (PnP) algorithm, which is flexible to different compression ratios. And the second decoder is a memory efficient end-to-end convolutional neural network, which aims for real-time decoding. Extensive results on the high definition images and videos demonstrate the superior performance of the proposed codec and the robustness against bit quantization.

Figures (8)

• Figure 1: Pipeline of the proposed Block Modulating Video Compression (BMVC) encoder. (a) For each input image with size of $N_h \times N_w$. The input frame is first divided into non-overlapping blocks of size $B_h\times B_w$. Then the image blocks are modulated (element-wise multiplication) by binary masks of the same size. Next, these modulated image blocks are summed together to yield a single block of size $B_h\times B_w$. At last, the summed modulated image block is quantized to a user-defined bit depth ($8\sim 16$-bit) and then transmitted to the receiver. An equivalent encoding pipeline is shown in (b) where the modulation happens before dividing into blocks.
• Figure 2: Plug-and-Play optimization-based decoding algorithm for Block Modulating Video Compression (BMVC-PnP). The encoded image block along with the modulation binary masks are fed into the BMVC-PnP decoder as inputs. The BMVC-PnP iteratively performs a linear projection step to account for the BMVC encoding process and a deep-learning-based denoising step as an implicit prior. We use a pre-trained FFDNet Zhang18TIP_FFDNet as the denoising CNN for its flexibility and robustness against various noise levels.
• Figure 3: End-to-End neural-network-based decoding algorithm for Block Modulating Video Compression (BMVC-E2E). The encoded image block along with the modulation binary masks are fed into the BMVC-E2E decoder as inputs. The feed-forward BMVC-E2E decoder consists of several stages, where each stage contains a linear projection step and a convolutional neural network. All BMVC-E2E decoders are trained in an end-to-end fashion. We use 2D-U-Net and 3D-CNN with reversible blocks (RevSCI) to facilitate memory-efficient training. Detailed network structures can be found in subsection \ref{['Sec:network']}.
• Figure 4: Test dataset (set13) we used to evaluate the BMVC pipeline and other compression methods.
• Figure 5: Decoded image results at various compression ratios with the proposed BMVC-PnP and BMVC-E2E approaches. The BMVC-E2E results are consistently of good decoding quality at both low and high Crs. The BMVC-PnP decoder provides higher image quality for low Crs while produces some denoising artefacts at high Crs.