Object Placement for Anything

TL;DR

This work tackles open-set object placement by addressing the limited labeled data problem through a semi-supervised framework that jointly leverages small labeled datasets and large unlabeled image collections. It introduces similarity variation transfer, enabling knowledge about how placement changes affect rationality to propagate from labeled to unlabeled data via a shared similarity classifier and adversarial domain alignment. The approach is complemented by constructing the Open Object Placement Assessment (OOPA) dataset from Open Images and extensive experiments showing improved generalization, plausibility, and diversity over state-of-the-art methods. Overall, the method demonstrates that incorporating unlabeled data with targeted loss components yields more robust and transferable object placement models for real-world scenes.

Abstract

Object placement aims to determine the appropriate placement (\emph{e.g.}, location and size) of a foreground object when placing it on the background image. Most previous works are limited by small-scale labeled dataset, which hinders the real-world application of object placement. In this work, we devise a semi-supervised framework which can exploit large-scale unlabeled dataset to promote the generalization ability of discriminative object placement models. The discriminative models predict the rationality label for each foreground placement given a foreground-background pair. To better leverage the labeled data, under the semi-supervised framework, we further propose to transfer the knowledge of rationality variation, \emph{i.e.}, whether the change of foreground placement would result in the change of rationality label, from labeled data to unlabeled data. Extensive experiments demonstrate that our framework can effectively enhance the generalization ability of discriminative object placement models.

Figures (4)

• Figure 1: The illustration of similarity transfer from labeled data to unlabeled data. The foregrounds are marked with red outlines. The red and green image borders denote rational and irrational placements, respectively. The numbers above the arrows indicate pairwise similarities of two placements, i.e., whether they have the same rationality label.
• Figure 2: Illustration of our semi-supervised object placement framework, which performs model training and label correction iteratively. In the step of model training, we use base object placement model to extract rationality features, which are sent to the supervised classifier. Pairs of rationality features are sent to similarity classifier and the intermediate similarity features are sent to domain classifier.
• Figure 3: The comparison of the rationality score maps predicted by FOPA model and our model. The foregrounds are highlighted with red outlines.
• Figure 4: Qualitative comparison with object placement baselines on OPA test set and OOPA test set. The foregrounds are highlighted with red outlines.