Making Images Real Again: A Comprehensive Survey on Deep Image Composition

TL;DR

This survey systematically catalogs the landscape of deep image composition, defining three core inconsistency types and decomposing the problem into sub-tasks such as object placement, image blending, image harmonization, and shadow/reflection generation. It distinguishes category-specific and instance-specific, and traditional versus deep-learning approaches, including a dedicated discussion of generative composition and foreground object search. The authors provide a comprehensive mapping of methods, datasets, and evaluation metrics, introduce public resources like the Awesome-Object-Insertion repo and the libcom toolbox, and offer practical guidance on method selection and experimental evaluation. Together, these contributions establish a roadmap for advancing realistic image composition across static images and beyond, with emphasis on diffusion-based generative approaches and integrative pipelines. The work serves as a foundational reference for researchers and developers designing end-to-end image composition systems for applications in creative media, advertising, and data augmentation.

Abstract

As a common image editing operation, image composition (object insertion) aims to combine the foreground from one image and another background image, to produce a composite image. However, there are many issues that could make the composite images unrealistic. These issues can be summarized as the inconsistency between foreground and background, which includes appearance inconsistency (e.g., incompatible illumination), geometry inconsistency (e.g., unreasonable size), and semantic inconsistency (e.g., mismatched semantic context). The image composition task could be decomposed into multiple sub-tasks, in which each sub-task targets one or more issues. Specifically, object placement aims to find reasonable scale, location, and shape for the foreground. Image blending aims to address the unnatural boundary between foreground and background. Image harmonization aims to adjust the illumination statistics of foreground. Shadow (resp., reflection) generation aims to generate plausible shadow (resp., reflection) for the foreground. These sub-tasks can be executed sequentially or in parallel to acquire realistic composite images. To the best of our knowledge, there is no previous survey on image composition. In this paper, we conduct a comprehensive survey over the sub-tasks and combined task of image composition. For each one, we summarize the existing methods, available datasets, and common evaluation metrics. Datasets and codes for image composition are summarized at https://github.com/bcmi/Awesome-Object-Insertion. We have also contributed the first image composition toolbox libcom: https://github.com/bcmi/libcom, which assembles 10+ image-composition-related functions (e.g., image blending, image harmonization, object placement, shadow generation, generative composition). The ultimate goal of this toolbox is to solve all image composition problems with simple `import libcom'.

Figures (20)

• Figure 1: Image composition aims to combine the foreground object and the background image to generate a realistic composite image.
• Figure 2: The quality of composite image is degraded by the appearance inconsistency, geometric inconsistency, and semantic inconsistency. Each type of inconsistency involves a number of issues. Each sub-task targets one or more issues.
• Figure 3: Previous works perform multiple sub-tasks (e.g., object placement, image blending, image harmonization, shadow/reflection generation) sequentially or in parallel to achieve the goal of image composition.
• Figure 4: Examples of unreasonable object placements. The inserted foreground objects are marked with red outlines. From left to right: (a) objects with inappropriate size; (b) unreasonable occlusion; (c) objects hanging in the air; (d) objects appearing at the semantically unreasonable place; (e) inconsistent perspectives.
• Figure 5: In the left subfigure, we compare category-specific object placement with instance-specific object placement. In the right subfigure, we show the taxonomy of existing object placement methods.