Prune and Repaint: Content-Aware Image Retargeting for any Ratio

TL;DR

By focusing on the content and structure of the foreground, the PruneRepaint approach adaptively avoids key content loss and deformation, while effectively mitigating artifacts with local repainting.

Abstract

Image retargeting is the task of adjusting the aspect ratio of images to suit different display devices or presentation environments. However, existing retargeting methods often struggle to balance the preservation of key semantics and image quality, resulting in either deformation or loss of important objects, or the introduction of local artifacts such as discontinuous pixels and inconsistent regenerated content. To address these issues, we propose a content-aware retargeting method called PruneRepaint. It incorporates semantic importance for each pixel to guide the identification of regions that need to be pruned or preserved in order to maintain key semantics. Additionally, we introduce an adaptive repainting module that selects image regions for repainting based on the distribution of pruned pixels and the proportion between foreground size and target aspect ratio, thus achieving local smoothness after pruning. By focusing on the content and structure of the foreground, our PruneRepaint approach adaptively avoids key content loss and deformation, while effectively mitigating artifacts with local repainting. We conduct experiments on the public RetargetMe benchmark and demonstrate through objective experimental results and subjective user studies that our method outperforms previous approaches in terms of preserving semantics and aesthetics, as well as better generalization across diverse aspect ratios. Codes will be available at https://github.com/fhshen2022/PruneRepaint.

Figures (6)

• Figure 1: An example to show bad cases such as deformation, content loss, discontinuity in lines, inconsistent results and a good case.
• Figure 2: The overall architecture of our proposed PruneRepaint. The input consists of a RGB image and a target ratio. The saliency map, obtained by saliency detection, is further introduced into content-aware seam-carving module for preliminary retargeting. The preliminary retargeted result is then processed by the adaptive repainting region determination module to identify the abrupt pixel regions that need to be repainted. Utilizing the original image as guidance, the image is inpainted with the image-guided repainting module, generating the final targeted image with target ratio.
• Figure 3: The architecture of the image-guided repainting module.
• Figure 4: Visual comparison to other retargeting methods on ratio 16:9.
• Figure 5: Visual comparison to other retargeting methods on ratio 9:16.