Block-Based Multi-Scale Image Rescaling

TL;DR

This work tackles the challenge that traditional image rescaling emphasizes a single global scaling rate, neglecting regional information content in high-resolution images. It proposes Block-Based Multi-Scale Image Rescaling (BBMR), comprising a Downscaling Module that assigns per-block downscale rates across 128×128 blocks while preserving the overall rate, and an Upscaling Module with JointSR that performs block-wise super-resolution and includes a deblocking branch to suppress cross-block artifacts. The method is validated with two SR backbones, OmniSR and CRAFT, on 2K and 4K datasets, showing PSNR gains of roughly 1.5 dB on average and up to 1.97 dB in some configurations, with modest increases in Upscaling FLOPs. A very lightweight SR model is also proposed to reduce server-side computation with only minor quality loss, and extensive ablations confirm the effectiveness of the block-based downscaling strategy, the deblock branch, and JointSR. Overall, BBMR offers a practical framework to improve SR quality for high-resolution image rescaling while controlling computational cost, enabling better storage/transmission efficiency for 2K/4K media.

Abstract

Image rescaling (IR) seeks to determine the optimal low-resolution (LR) representation of a high-resolution (HR) image to reconstruct a high-quality super-resolution (SR) image. Typically, HR images with resolutions exceeding 2K possess rich information that is unevenly distributed across the image. Traditional image rescaling methods often fall short because they focus solely on the overall scaling rate, ignoring the varying amounts of information in different parts of the image. To address this limitation, we propose a Block-Based Multi-Scale Image Rescaling Framework (BBMR), tailored for IR tasks involving HR images of 2K resolution and higher. BBMR consists of two main components: the Downscaling Module and the Upscaling Module. In the Downscaling Module, the HR image is segmented into sub-blocks of equal size, with each sub-block receiving a dynamically allocated scaling rate while maintaining a constant overall scaling rate. For the Upscaling Module, we introduce the Joint Super-Resolution method (JointSR), which performs SR on these sub-blocks with varying scaling rates and effectively eliminates blocking artifacts. Experimental results demonstrate that BBMR significantly enhances the SR image quality on the of 2K and 4K test dataset compared to initial network image rescaling methods.

Figures (5)

• Figure 1: Comparison of traditional image rescaling methods (left) and our block-based multi-scale image rescaling framework (right) at ×4 scaling rate. Both methods use bicubic interpolation downscaling and OmniSR model upscaling. The improvement in the image quality of the red box is due to the smaller scaling rate.
• Figure 2: The overall framework structure of the proposed BBMR when the scaling rate is ×2, ×4 and ×8. Downscaling Module: Outputs LR image sub-blocks with three different resolutions through block-based multi-scale downscaling strategy. Upscaling Module: Generates high-quality SR images using the JointSR method. Upsample contains a convolution layer and a pixelshuffle module.
• Figure 3: Illustration of the block edge replacement policy and deblock branch structure.
• Figure 4: Comparison of visual quality between the BBMR method and traditional image rescaling method using different downscaling and upscaling approachs.
• Figure 5: Visual quality comparison of block effects using bicubic downscaling. The boundary of the image block is located at the cross in the middle of the image. The first row shows the upscaling results of OmniSR, while the second row shows the upscaling results of CRAFT.