Protecting NeRFs' Copyright via Plug-And-Play Watermarking Base Model

TL;DR

The paper tackles the challenge of copyright protection for NeRF-based content by introducing NeRFProtector, a plug-and-play watermarking framework that embeds ownership messages into NeRFs during creation using a pre-trained watermarking base model. It employs a Progressive Global Rendering scheme to distill watermark knowledge into the NeRF across multiple rendering scales, preserving the original scene representation while enabling reliable watermark extraction from rendered views. Key contributions include building a HiDDeN-based watermarking base model, a progressive distillation strategy, and demonstrated compatibility with several NeRF variants, along with ablation and threat analyses. The approach achieves robust bit extraction with minimal impact on reconstruction quality and substantially reduces embedding time compared with prior art, providing a practical IP-protection tool for NeRF-based workflows across diverse pipelines.

Abstract

Neural Radiance Fields (NeRFs) have become a key method for 3D scene representation. With the rising prominence and influence of NeRF, safeguarding its intellectual property has become increasingly important. In this paper, we propose \textbf{NeRFProtector}, which adopts a plug-and-play strategy to protect NeRF's copyright during its creation. NeRFProtector utilizes a pre-trained watermarking base model, enabling NeRF creators to embed binary messages directly while creating their NeRF. Our plug-and-play property ensures NeRF creators can flexibly choose NeRF variants without excessive modifications. Leveraging our newly designed progressive distillation, we demonstrate performance on par with several leading-edge neural rendering methods. Our project is available at: \url{https://qsong2001.github.io/NeRFProtector}.

Figures (8)

• Figure 1: Proposed scenario of our method. NeRF creators can obtain a pre-trained watermarking extractor from a third party (e.g., open-source library) or train a message extractor separately via standard pipelines, as the watermarking base model$\mathcal{F}$. Once obtained, this base model is considered "plug-and-play" in our scenario. Throughout the NeRF creation process, creators can readily use this base model to embed watermarks in their NeRF. After the optimization of NeRF is complete, they obtain a watermarked NeRF. When this watermarked NeRF is distributed publicly, the creators can then use the same base model to retrieve binary messages from newly rendered views, thereby asserting their ownership.
• Figure 2: Our plug-and-play method to watermark NeRF during its creation. (1) Building watermarking base model: The watermarking base model $\mathcal{F}$ can be sourced from a third party (\ref{['sec:pre-training']}). We implement a HiDDeN zhu2018hidden framework to get the pre-trained message extractor as our watermarking base model $\mathcal{F}$. During the training, the encoder $\mathcal{E}$ encodes a randomly selected 48-kit message $\mathbf{m}$ and cover image $\mathbf{I}_o$ and outputs a watermarked image $\mathbf{I}_{en}$. Then the message extractor $\mathcal{F}$ extracts embedded message $\hat{\mathbf{m}}$ from the watermarked image. Message encoder $\mathcal{E}$ is discarded after building watermarking base model $\mathcal{F}$. (2) NeRF creation with message distillation: NeRF creators first fix a copyright message $\mathbf{m}$, then employ this base model $\mathcal{F}$ to embed selected watermarks to NeRFs during the creation process via Progressive Global Rendering (PGR) and message distillation (\ref{['sec:distilling']}). When the optimization of NeRF is finalized, creators immediately obtain a watermarked NeRF. (3) Extracting watermark: Subsequently, they can utilize the base model $\mathcal{F}$ to retrieve binary watermarks from the rendered images, asserting their ownership.
• Figure 3: Visual quality comparisons with CopyRNeRF luo2023copyrnerf. The texts below the images show the results of PSNR and bit accuracy. We show the differences $\times 10$ between the rendered views and the ground truth for both methods. Our NeRFProtector exhibits better consistency across multiple viewpoints, achieving a well-balanced trade-off between reconstruction quality and bit accuracy.
• Figure 4: Image-level distortion experiment on common image distortions on rendered views to determine if the extractor can still extract the message. Operations like sharpness, cropping, contrast, and text overlay have little impact on message extraction. Even after severe operations (JPEG and Resize), our method still maintains satisfactory accuracy ($\geq 70\%$).
• Figure 5: Visual heat map comparisons of local rendering and progressive global rendering. The text below denotes PSNR/Bit accuracy. The colors represent the area's impact on message extraction. Our PGR can distill messages into NeRF with high accuracy.