AEGIS: Preserving privacy of 3D Facial Avatars with Adversarial Perturbations

TL;DR

This work tackles biometric privacy for photorealistic 3D avatars by introducing AEGIS, which perturbs only the DC coefficients $\mathcal{C}$ of spherical harmonics in 3D Gaussian Avatars using constrained PGD to minimize the expected embedding similarity $s(\mathbf{e}_a(\mathcal{C},\mathbf{v}),\mathbf{e}_r)$ across a viewpoint distribution $\mathbf{T}$ while keeping $\|\mathcal{C}-\mathcal{C}^0\|_\infty \le \epsilon$. The method preserves geometry and view-dependent appearance via differentiable rendering, and uses RetinaFace alignment with ArcFace/AdaFace embeddings for verification guidance. Results show complete de-identification (0% Rank-1/Match) at moderate budgets (e.g., $\epsilon=0.2$ for ArcFace and $\epsilon=0.1$ for AdaFace) with high perceptual fidelity (SSIM around 0.956 and PSNR around 35.5 dB) and preservation of soft traits like age, race, gender, and emotion. The work demonstrates the practicality of geometry-aware, view-consistent privacy for immersive 3D avatars, enabling privacy-preserving use in video calls and virtual environments without compromising realism or usability.

Abstract

The growing adoption of photorealistic 3D facial avatars, particularly those utilizing efficient 3D Gaussian Splatting representations, introduces new risks of online identity theft, especially in systems that rely on biometric authentication. While effective adversarial masking methods have been developed for 2D images, a significant gap remains in achieving robust, viewpoint-consistent identity protection for dynamic 3D avatars. To address this, we present AEGIS, the first privacy-preserving identity masking framework for 3D Gaussian Avatars that maintains the subject's perceived characteristics. Our method aims to conceal identity-related facial features while preserving the avatar's perceptual realism and functional integrity. AEGIS applies adversarial perturbations to the Gaussian color coefficients, guided by a pre-trained face verification network, ensuring consistent protection across multiple viewpoints without retraining or modifying the avatar's geometry. AEGIS achieves complete de-identification, reducing face retrieval and verification accuracy to 0%, while maintaining high perceptual quality (SSIM = 0.9555, PSNR = 35.52 dB). It also preserves key facial attributes such as age, race, gender, and emotion, demonstrating strong privacy protection with minimal visual distortion.

Figures (12)

• Figure 1: Photos enable 2D face verification but cannot pass 3D verification with liveness detection. Since 3D avatars can be used in such systems, their identity must be protected. Unlike 2D masking, which must be reapplied for each pose, AEGIS achieves consistent identity protection across different viewpoints and poses through geometry-aware 3D masking.
• Figure 2: An overview of the AEGIS identity masking pipeline. The process adversarially optimizes the color parameters $\mathcal{C}^t$ of a 3D Gaussian avatar $\mathcal{G}$ to evade face recognition by the model $F(\cdot)$. A fixed reference embedding $\mathbf{e}_r$ is first obtained by rendering the original avatar $\mathcal{G}_0$ under canonical camera $\mathbf{v}_r$ and pose $\mathbf{p}_r$ parameters, and passing the result through $F(\cdot)$. During each PGD optimization step, a set of camera parameters $\{ \mathbf{v}_k \}_{k=1}^K$ is sampled to capture diverse viewpoints. The updated avatar $\mathcal{G}_t$ is rendered from these viewpoints using the rendering function $R(\cdot)$ and alignment module $A(\cdot)$, producing a batch of images. Identity embeddings $\{\mathbf{e}_k\}$ are then extracted using $F(\cdot)$, and their average cosine similarity $\bar{s}$ with the reference embedding $\mathbf{e}_r$ is computed. This similarity defines the logits ($\bar{s}$ for "match" and $-\bar{s}$ for "no match"), from which a cross-entropy loss is computed while targeting the "no match" class. The resulting loss is backpropagated through the network using PGD to update the color parameters $\mathcal{C}^t$, yielding an adversarial 3D representation.
• Figure 3: Visualization of example identity masks obtained using AEGIS (for AdaFace) and reference 2D masking methods.
• Figure 4: Effect of pose variation and privacy budget $\epsilon$ on identity masking persistence (against AdaFace verification system). Top: unmasked avatar with high similarity during verification across poses. Middle: masked avatar with $\epsilon=0.1$ shows reduced similarity but residual identity cues at some angles. Bottom: with $\epsilon=0.2$, avatar is completely de-identified across all poses. These results demonstrate the trade-off between visual fidelity and privacy strength, as higher $\epsilon$ values yield stronger privacy at the cost of greater perceptual deviation.
• Figure 5: Qualitative comparison of adversarial perturbations applied to different components of the 3D Gaussian avatar representation. Each row shows the resulting masked avatar under attacks targeting AC coefficients, opacity, position, rotation, and scale, contrasted with the best-performing AEGIS configuration.