GeoRemover: Removing Objects and Their Causal Visual Artifacts

TL;DR

This paper tackles the challenge of removing an object and its causal visual artifacts (such as shadows and reflections) by decoupling object removal into geometry removal and appearance rendering. It introduces a two-stage framework where the first stage edits the scene geometry (depth) under strictly mask-aligned supervision, and the second stage renders an RGB image conditioned on the updated geometry to implicitly eliminate artifacts. A Direct Preference Optimization (DPO) approach guides geometry completion by promoting smooth depth flow within the masked region, reducing hallucinations and over-editing. The method demonstrates state-of-the-art performance on RemovalBench, RORD-Val, and CausRem, with quantitative gains in FID, CMMD, LPIPS, PSNR, and IoU for artifact removal, as well as qualitative improvements in preserving unmasked content. This geometry-guided, bidirectional rendering framework offers a controllable and robust solution for realistic object removal in complex scenes, with broad implications for AR/VR content editing and image restoration.

Abstract

Towards intelligent image editing, object removal should eliminate both the target object and its causal visual artifacts, such as shadows and reflections. However, existing image appearance-based methods either follow strictly mask-aligned training and fail to remove these causal effects which are not explicitly masked, or adopt loosely mask-aligned strategies that lack controllability and may unintentionally over-erase other objects. We identify that these limitations stem from ignoring the causal relationship between an object's geometry presence and its visual effects. To address this limitation, we propose a geometry-aware two-stage framework that decouples object removal into (1) geometry removal and (2) appearance rendering. In the first stage, we remove the object directly from the geometry (e.g., depth) using strictly mask-aligned supervision, enabling structure-aware editing with strong geometric constraints. In the second stage, we render a photorealistic RGB image conditioned on the updated geometry, where causal visual effects are considered implicitly as a result of the modified 3D geometry. To guide learning in the geometry removal stage, we introduce a preference-driven objective based on positive and negative sample pairs, encouraging the model to remove objects as well as their causal visual artifacts while avoiding new structural insertions. Extensive experiments demonstrate that our method achieves state-of-the-art performance in removing both objects and their associated artifacts on two popular benchmarks. The code is available at https://github.com/buxiangzhiren/GeoRemover.

Figures (12)

• Figure 1: Comparison of object removal training paradigms. (a) Strictly mask-aligned training edits only masked regions but leaves causal visual artifacts (shadow) unaddressed. (b) Loosely mask-aligned training allows broader context-aware corrections but lacks clear guidance, leading to confusion and uncontrollable edits. (c) Our method decouples geometry and appearance for object removal: we first edit the scene geometric representation (in the form of a depth map) under strictly mask-aligned supervision, then render a realistic image where both objects and causal visual artifacts (shadow) are cleanly removed.
• Figure 2: The training framework of Stage 1: Geometry Removal. Given an input image and object mask, we first estimate the geometric representation (in the form of a depth map) and construct a masked geometry input. The masked depth map, together with the mask, is then fed into a diffusion model to predict the edited geometry. To discourage structure insertion and encourage object removal, we construct two geometry completion paths: a positive path where the object is successfully removed with smooth depth flow, and a negative path where the object remains with sharp depth transitions. The model is trained to prefer the positive path and suppress the negative one.
• Figure 3: (a) We compare model outputs trained with and without our DPO objective (i.e., $\mathcal{L}_{\text{BT}}$). Without DPO, the model often inserts or retains undesired content in the masked region, leading to unrealistic geometry. With DPO, the model learns to prefer geometry completions that successfully remove the object while preserving surrounding structures. (b) The training framework of Stage 2. Given geometry-aware conditions (e.g., depth maps), we train a diffusion model to perform image translation for both object removal and insertion. More details can be found in Appendix \ref{['sec:appendstage2']}.
• Figure 4: Comparison between our one-stage and two-stage object removal strategies. Two-stage design improves edit quality by separating geometry reasoning from appearance generation.
• Figure 5: Depth errors caused by motion blur result in removal failure. Applying a simple Fill-in strategy within the mask restores geometric contrast and yields correct removal.