SSH-Net: A Self-Supervised and Hybrid Network for Noisy Image Watermark Removal

TL;DR

SSH-Net addresses noisy image watermark removal without requiring paired clean references by introducing self-supervised watermark-free synthesis and a dual-path network that separates a lightweight CNN denoiser from a CNN-Transformer hybrid that handles watermark and noise jointly. A shared encoder feeds both branches, whose outputs are fused by a gating mechanism to produce a final clean image, with training guided by a mixed loss combining structure and perceptual texture terms. The key innovations—the Sparse Transformer U-Net for the watermark-and-noise path, the shared encoder, and the adaptive FFU gate—yield superior PSNR, SSIM, and LPIPS across varying noise and watermark levels while reducing computational cost relative to fully CNN-based dual nets. These contributions enable practical, robust watermark removal in real-world scenarios where reference watermark-free images are unavailable.

Abstract

Visible watermark removal is challenging due to its inherent complexities and the noise carried within images. Existing methods primarily rely on supervised learning approaches that require paired datasets of watermarked and watermark-free images, which are often impractical to obtain in real-world scenarios. To address this challenge, we propose SSH-Net, a Self-Supervised and Hybrid Network specifically designed for noisy image watermark removal. SSH-Net synthesizes reference watermark-free images using the watermark distribution in a self-supervised manner and adopts a dual-network design to address the task. The upper network, focused on the simpler task of noise removal, employs a lightweight CNN-based architecture, while the lower network, designed to handle the more complex task of simultaneously removing watermarks and noise, incorporates Transformer blocks to model long-range dependencies and capture intricate image features. To enhance the model's effectiveness, a shared CNN-based feature encoder is introduced before dual networks to extract common features that both networks can leverage. Our code will be available at https://github.com/wenyang001/SSH-Net.

Figures (5)

• Figure 1: The overall architecture of the proposed Self-Supervised and Hybrid Network (SSH-Net), which mainly contains a shared encoder, a noise removal decoder (NRD), a watermark and noise removal decoder (WNRD), and a feature fusion unit (FFU). SSH-Net leveraged the supervised and self-supervised technique to generate groud-truth samples $(X_{\text{w}}, Y_{\text{w}})$ to optimize the model.
• Figure 2: The overall architecture of the proposed Sparse Transformer U-Net, which follows a 3-level Transformer U-Net design. Each level comprises multiple Sparse Transformer Blocks (STBs) to process features at different scales. Each STB includes a Sparse Self-Attention (SSA) mechanism and a Fully-Connected Network (FFN).
• Figure 3: Results comparison trained under a specific transparency and a specific noise condition ($\delta$ = 25 and transparency = 0.3). (a) Ground Truth (b) 20.19 dB (c) 28.48 dB (d) 29.89 dB (e) 29.68 dB (f) 31.39 dB (g) 32.15 dB (h) 32.26 dB.
• Figure 4: Results comparison trained under a specific noise and a specific transparency condition ($\delta$ = 15 and transparency = 0.3). (a) Ground Truth (b) 24.04 dB (c) 27.44 dB (d) 27.19 dB (e) 30.16 dB (f) 29.23 dB (g) 31.54 dB (h) 31.95 dB.
• Figure 5: Visualizations of the gating mechanism in the proposed Feature Fusion Unit (FFU).