Bridging Knowledge Gap Between Image Inpainting and Large-Area Visible Watermark Removal

TL;DR

This work tackles the challenging problem of large-area visible watermark removal by bridging image inpainting and watermark restoration through a feature-adapting framework. It introduces two branches, Watermark Component Cleaning (WCC) and Background Content Embedding (BCE), to supply residual background prompts to a pre-trained LaMa inpainting backbone via gated fusion modules. The model is trained with coarse watermark masks to reduce dependence on precise segmentation, and it demonstrates state-of-the-art performance on the large ILAW dataset and a real-world dataset, indicating robustness to mask quality and watermark scale. The approach offers a practical path for robust watermark removal in real-world editing and for evaluating watermark resilience in diverse conditions, with code to be released in supplementary materials.

Abstract

Visible watermark removal which involves watermark cleaning and background content restoration is pivotal to evaluate the resilience of watermarks. Existing deep neural network (DNN)-based models still struggle with large-area watermarks and are overly dependent on the quality of watermark mask prediction. To overcome these challenges, we introduce a novel feature adapting framework that leverages the representation modeling capacity of a pre-trained image inpainting model. Our approach bridges the knowledge gap between image inpainting and watermark removal by fusing information of the residual background content beneath watermarks into the inpainting backbone model. We establish a dual-branch system to capture and embed features from the residual background content, which are merged into intermediate features of the inpainting backbone model via gated feature fusion modules. Moreover, for relieving the dependence on high-quality watermark masks, we introduce a new training paradigm by utilizing coarse watermark masks to guide the inference process. This contributes to a visible image removal model which is insensitive to the quality of watermark mask during testing. Extensive experiments on both a large-scale synthesized dataset and a real-world dataset demonstrate that our approach significantly outperforms existing state-of-the-art methods. The source code is available in the supplementary materials.

Figures (9)

• Figure 1: (a) Existing multi-task based methods such as cun2021splitliang2021visiblesun2023denet adopt a shared encoder and multi-branch decoder for implementing sub-tasks such as watermark segmentation, watermark decomposition, and background restoration; (b) We propose a novel solution through adapting an image inpainting backbone with prompt information extracted from a watermark component cleaning and a background content embedding branches. Moreover, we relieve the dependence on high-quality watermark masks by leveraging coarse masks to guide the inference process.
• Figure 2: The first column showcases two input examples which are covered by large-area watermarks. Though coarse watermark masks (second column) are available, our method (fourth column) can effectively remove these watermarks and accurately recover the background, showing significant superiority over SplitNet (third column).
• Figure 3: Overview of our framework which adapts an image inpainting backbone model, LaMa, to address the visible watermark removal task. Given an input image $\mathbf X$ and a coarse mask $\mathbf M$, the watermark component cleaning branch (WCC) is employed to preclude the interference information brought by watermarks from the input image. Then, a background content embedding branch (BCE) is used to extract prompt features from the background component image and the original input image. We enhance the intermediate features of LaMa with these feature extracted by WCC and BCE branches.
• Figure 4: Illustration of the gated fusion module (GFM).
• Figure 5: Watermark segmentation generated by blind visible watermark removal methods SLBR and SplitNet.