Splats in Splats: Robust and Effective 3D Steganography towards Gaussian Splatting

TL;DR

Splats in splats introduces the first 3D Gaussian Splatting (3DGS) steganography framework that embeds 3D content directly into vanilla 3DGS without altering its attributes, enabling provenance verification without sacrificing usability. The method leverages an importance-graded SH coefficient encryption strategy and autoencoder-assisted opacity mapping to hide 3D content within the original 3DGS representation, preserving rendering speed and the original pipeline. Empirical results show state-of-the-art fidelity and efficiency, with about 100 FPS rendering and robust performance against noise and pruning attacks, while providing a private-key mechanism for content extraction. This work advances practical copyright protection and provenance verification for 3DGS assets, with implications for secure 3D content generation and management across applications.

Abstract

3D Gaussian splatting (3DGS) has demonstrated impressive 3D reconstruction performance with explicit scene representations. Given the widespread application of 3DGS in 3D reconstruction and generation tasks, there is an urgent need to protect the copyright of 3DGS assets. However, existing copyright protection techniques for 3DGS overlook the usability of 3D assets, posing challenges for practical deployment. Here we describe splats in splats, the first 3DGS steganography framework that embeds 3D content in 3DGS itself without modifying any attributes. To achieve this, we take a deep insight into spherical harmonics (SH) and devise an importance-graded SH coefficient encryption strategy to embed the hidden SH coefficients. Furthermore, we employ a convolutional autoencoder to establish a mapping between the original Gaussian primitives' opacity and the hidden Gaussian primitives' opacity. Extensive experiments indicate that our method significantly outperforms existing 3D steganography techniques, with 5.31% higher scene fidelity and 3x faster rendering speed, while ensuring security, robustness, and user experience.

Figures (8)

• Figure 1: Left: GS-Hider and our method's rendering pipeline. GS-Hider zhang2024gshider employs a coupled feature field and neural decoders to render the original and hidden scenes simultaneously, affecting user’s standard utilization. We retain the vanilla 3DGS pipeline to preserve user experience. Right: Comparison of different 3DGS steganography methods. Existing works all have shortcomings in terms of robustness, fidelity, efficiency, and usability. Our method can maximize the fidelity of the original scene while ensuring the rendering speed and usability of 3DGS, as well as the security of the steganography.
• Figure 2: Overview of the Framework. To ensure a seamless user experience, we preserves the same attributes as the vanilla 3DGS, while enabling the extraction of embedded 3D content for owners. The information embedding and extraction process involves three key steps: a) Hidden attributes training: We utilize the original and hidden views to train two sets of SH coefficients and opacity, while ensuring that both sets share the same Gaussian primitive locations. b) Importance-graded SH coefficient encryption/decryption: We prioritize SH coefficients by significance, embedding more important hidden SH coefficients into higher-order components of the original SH via bit-shifting, aiming to achieve superior rendering fidelity and high robustness against noise attack. c) Autoencoder-assisted opacity mapping: We apply a threshold to discard trivial hidden opacities, subsequently employing an convolutional autoencoder to model the transformation from original to hidden opacity.
• Figure 3: Importance of Spherical Harmonics order. We retain the specified order of SH and render the image while setting other orders to zero. Higher-order SH contains only a small amount of High-frequency information.
• Figure 4: Qualitative comparisons on Mipnerf360 datasets. Our method achieves sharper results with more consistent structures.
• Figure 5: Comparison of rendering results. GS-hider and 3DGS+SteganeNeRF have obvious information leakage. Better viewed on screen with zoom in.