Task-Decoupled Image Inpainting Framework for Class-specific Object Remover

TL;DR

A task-decoupled image inpainting framework is proposed which generates two separate inpainting models: an object restorer for object restoration tasks and an object remover for object removal tasks, and a class-specific object remover, aiming to better erase target class objects than general object removers.

Abstract

Object removal refers to the process of erasing designated objects from an image while preserving the overall appearance. Existing works on object removal erase removal targets using image inpainting networks. However, image inpainting networks often generate unsatisfactory removal results. In this work, we find that the current training approach which encourages a single image inpainting model to handle both object removal and restoration tasks is one of the reasons behind such unsatisfactory result. Based on this finding, we propose a task-decoupled image inpainting framework which generates two separate inpainting models: an object restorer for object restoration tasks and an object remover for object removal tasks. We train the object restorer with the masks that partially cover the removal targets. Then, the proposed framework makes an object restorer to generate a guidance for training the object remover. Using the proposed framework, we obtain a class-specific object remover which focuses on removing objects of a target class, aiming to better erase target class objects than general object removers. We also introduce a data curation method that encompasses the image selection and mask generation approaches used to produce training data for the proposed class-specific object remover. Using the proposed curation method, we can simulate the scenarios where an object remover is trained on the data with object removal ground truth images. Experiments on multiple datasets show that the proposed class-specific object remover can better remove target class objects than object removers based on image inpainting networks.

Figures (9)

• Figure 1: Two tasks of an image inpainting network. Lama suvorov2022resolution is utilized for inpainting.
• Figure 2: The class-specific object restorer training process.
• Figure 3: Inpainting results of the inpainting network (Lama suvorov2022resolution) and the proposed class-specific object restorer. Car class is set as a target class.
• Figure 4: Input masked images. (a) shows an input image of a class-wise object removal task. (b) demonstrates a masked image generated using the proposed data curation method.
• Figure 5: Class-specific object remover training process.