GeometrySticker: Enabling Ownership Claim of Recolorized Neural Radiance Fields

TL;DR

GeometrySticker tackles ownership protection for recolorized Neural Radiance Fields (NeRFs) by embedding binary messages into geometry rather than colors. It attaches a message sticker to surface-proximal 3D points using a learnable Laplace CDF to select cover media and a simple addition $ ilde{σ}=σ+ψ m$, with recovery via a CNN-based extractor $D_{χ}$ trained alongside multiple losses to balance invisibility, recoverability, and ubiquity. The approach is implemented in PyTorch and demonstrated to be compatible with vanilla NeRF, InstantNGP, and TensoRF, preserving rendering quality while enabling ownership verification even after recolorization through CLIP-based, palette-based, or image-level color edits. Experimental results on Blender and LLFF show high bit accuracy (approximately $99\%$) and robust retrieval across recolorization types, outperforming prior watermarking methods and demonstrating scalability to different 3D representations. Overall, GeometrySticker provides scalable, view-consistent ownership protection for recolorized NeRFs with practical implications for safeguarding digital 3D assets.

Abstract

Remarkable advancements in the recolorization of Neural Radiance Fields (NeRF) have simplified the process of modifying NeRF's color attributes. Yet, with the potential of NeRF to serve as shareable digital assets, there's a concern that malicious users might alter the color of NeRF models and falsely claim the recolorized version as their own. To safeguard against such breaches of ownership, enabling original NeRF creators to establish rights over recolorized NeRF is crucial. While approaches like CopyRNeRF have been introduced to embed binary messages into NeRF models as digital signatures for copyright protection, the process of recolorization can remove these binary messages. In our paper, we present GeometrySticker, a method for seamlessly integrating binary messages into the geometry components of radiance fields, akin to applying a sticker. GeometrySticker can embed binary messages into NeRF models while preserving the effectiveness of these messages against recolorization. Our comprehensive studies demonstrate that GeometrySticker is adaptable to prevalent NeRF architectures and maintains a commendable level of robustness against various distortions. Project page: https://kevinhuangxf.github.io/GeometrySticker/.

Figures (11)

• Figure 1: Our proposed scenario for ownership claim over the recolorized NeRF. Users can construct their NeRF models using readily available platforms, such as NeRFStudio. Ownership message attachment: They can then swiftly stick binary messages onto those created NeRF models via our proposed GeometrySticker. These watermarked NeRF models remain suitable for standard recolorization processes (herein termed Authorized recolorization). Ownership verification: Should unauthorized recolorization occur, the creators of the NeRF models can retrieve the watermarks from the altered models to verify ownership.
• Figure 1: Illustration of the uniqueness of our method. The first row illustrates the NeRF model owner Bob claims the ownership by using CopyRNeRF luo2023copyrnerf or StegaNeRF li2023steganerf. However, when a malicious user applies unauthorized recolorization on Bob's model, the hidden ownership information can be corrupted and mismatch the original secret messages. The second row illustrates the NeRF model owner Alice claims the ownership by GeometrySticker. The NeRF model watermarked by GeometryStricker can be robust to different recolorizations. Even if a malicious user applies unauthorized recolorization on Alice's model, the hidden ownership information can still be reliably extracted and match the original secret messages.
• Figure 2: The framework of our proposed GeometrySticker. We employ a Multilayer Perceptron (MLP), denoted as $\theta_{m}$ to convert binary messages into a format that aligns with NeRF's geometry representation. Following this, a learnable Cumulative Distribution Function (CDF) is utilized to select 3D points close to the object surfaces with high geometry values as the cover medium. Subsequently, an addition is applied to attach the message (Equation (\ref{['eq:encode_watermark']})) onto the chosen cover media. Through this message attachment process, a watermarked NeRF is generated, capable of retaining its efficacy across diverse recolorizations. Should any unauthorized changes to color attributes occur, NeRF owners can retrieve the integrated watermarks to assert their ownership. $\theta_{\sigma}$ and $\theta_c$ indicate the representation for geometry and color.
• Figure 2: Message attachment into NeRF geometry values by applying Laplace CDF with different deviation parameters. The geometry values distribution is modeled by a Laplace distribution with the mean $\mu$ and deviation $\beta$. (a) indicates directly attaching messages on all geometry values can cause obvious distortion. (b) indicates applying Laplace CDF with fixed $\mu$ and $\beta$ can reduce perturbation but still show noticeable distortion. (c) indicates applying Laplace CDF with a learnable deviation parameter $\beta$ can find an optimized threshold for filtering 3D points and make the distortion imperceivable.
• Figure 3: Recolorized samples from our selected approaches. From left to right: original image, the result obtained via pallette-based recolorization Kuang_Luan_Bi_Shu_Wetzstein_Sunkavalli_2022 by changing the green base color to blue, the CLIP-based recolorization Wang_Chai_He_Chen_Liao_2022 result by giving "red" text prompt, and the color-jittering result by changing hue.