Image Super-Resolution Using T-Tetromino Pixels

TL;DR

This work proposes a novel binning concept using tetromino-shaped pixels and shows that using a small repeating cell consisting of only four T-tetrominoes is sufficient to achieve a higher image quality after upscaling.

Abstract

For modern high-resolution imaging sensors, pixel binning is performed in low-lighting conditions and in case high frame rates are required. To recover the original spatial resolution, single-image super-resolution techniques can be applied for upscaling. To achieve a higher image quality after upscaling, we propose a novel binning concept using tetromino-shaped pixels. It is embedded into the field of compressed sensing and the coherence is calculated to motivate the sensor layouts used. Next, we investigate the reconstruction quality using tetromino pixels for the first time in literature. Instead of using different types of tetrominoes as proposed elsewhere, we show that using a small repeating cell consisting of only four T-tetrominoes is sufficient. For reconstruction, we use a locally fully connected reconstruction (LFCR) network as well as two classical reconstruction methods from the field of compressed sensing. Using the LFCR network in combination with the proposed tetromino layout, we achieve superior image quality in terms of PSNR, SSIM, and visually compared to conventional single-image super-resolution using the very deep super-resolution (VDSR) network. For PSNR, a gain of up to \SI[retain-explicit-plus]{+1.92}{dB} is achieved.

Figures (9)

• Figure 1: Illustration of a conventional low-resolution binning process and the proposed tetromino binning. In both cases, less noise and higher frame rates are possible.
• Figure 2: Illustration of different sensor layouts, pixel shapes, and cell sizes. Blue lines indicate the boundaries of the pixels. Dark gray color is used for a single complete cell. The coherence is given for an image of size $M{\times}N=30{\times}30$ pixels.
• Figure 3: Two slices ($i=0$ and $i=1$) through the measurement matrices for the (a) low-resolution sensor, (b) Tetromino sensor from Galdo2014, and (c) the proposed $4{\times}4$ T-tetromino sensor.
• Figure 4: Ice graph of the $4{\times}4$ T-tetromino sensor from Figure \ref{['fig:tetris_sensor_vs_lr_skizze']} (c). The rules for any valid ice graph are shown on the right side up to rotational and mirror symmetry.
• Figure 5: Ice graph of a randomly generated $8{\times}8$ T-tetromino sensor. Other than the previous tetromino tilings, this tiling itself does not show any cell boundaries since the pixels are tightly geared into each other after periodic repetition.