Geometry Cloak: Preventing TGS-based 3D Reconstruction from Copyrighted Images

TL;DR

This work proposes a novel image protection approach that embeds invisible geometry perturbations, termed"geometry cloaks", into images before supplying them to TGS, and forces TGS to fail the 3D reconstruction in a specific way - by generating an identifiable customized pattern that acts as a watermark.

Abstract

Single-view 3D reconstruction methods like Triplane Gaussian Splatting (TGS) have enabled high-quality 3D model generation from just a single image input within seconds. However, this capability raises concerns about potential misuse, where malicious users could exploit TGS to create unauthorized 3D models from copyrighted images. To prevent such infringement, we propose a novel image protection approach that embeds invisible geometry perturbations, termed "geometry cloaks", into images before supplying them to TGS. These carefully crafted perturbations encode a customized message that is revealed when TGS attempts 3D reconstructions of the cloaked image. Unlike conventional adversarial attacks that simply degrade output quality, our method forces TGS to fail the 3D reconstruction in a specific way - by generating an identifiable customized pattern that acts as a watermark. This watermark allows copyright holders to assert ownership over any attempted 3D reconstructions made from their protected images. Extensive experiments have verified the effectiveness of our geometry cloak. Our project is available at https://qsong2001.github.io/geometry_cloak.

Figures (18)

• Figure 1: Overview of our scenario. (a) Images without protection. Images can be easily reconstructed into 3D models by malicious users with TGS zou2023tgs, posing a threat to the copyright of the image owner. (b) Digital Watermarking offers a solution by embedding copyright messages into the view-image before 3D reconstruction. However, the embedded message cannot be extracted from novel rendered views. (c) Geometry Cloak. Our geometry cloak utilizes the disturbance-prone components of TGS, achieving view-specific watermark embedding. Our method can compromise the unauthorized reconstructed 3D model while providing a verifiable pattern for copyright claim.
• Figure 2: Overall of our proposed method. We propose to induce the 3D reconstruction process with our geometry cloak. (a) The core representation of TGS zou2023tgs includes an explicit point cloud and an implicit triplane-based feature field. The features of the novel view image are extracted through the coordinates in the point cloud. (b) The target patterns (\ref{['sub1']}) are designed to induce the final reconstruction result. (c) In order to make the reconstruction result show some distinguishable characteristics, we use projected gradient descent (PGD) madry2017adv_training to iteratively optimize the reconstructed point cloud so that it has consistent characteristics with the target point cloud (\ref{['sub2']}).
• Figure 3: Two different target geometry patterns. (1) Pre-defined patterns: we directly convert alphanumeric characters into a 2D point cloud as watermarks. (2) Customized patterns: In $E_1$, we first extract the point cloud of the image that needs to be protected. In $E_2$, we edit the acquired point cloud through text-guided methods like instructP2P xu2023instructp2p or open-source software meshlab meshlab.
• Figure 4: Example of View-specific PGD. We use a 2D point cloud pre-defined pattern $\mathcal{P}_\text{w}$ as the target geometry pattern. The watermark is embedded at the viewing direction $\theta = xy$.
• Figure 5: Qualitative reconstructed results with different perturbing strategies. Compare to Gauss. noise and Adv. image, our method can significantly affect the reconstructed results, indicating the explicit geometry features are perturbation-prone during 3D reconstruction.