Black-box Adversaries from Latent Space: Unnoticeable Attacks on Human Pose and Shape Estimation

TL;DR

This work addresses security vulnerabilities in expressive human pose and shape (EHPS) estimation by proposing Unnoticeable Black-box Attack (UBA), a query-efficient method that perturbs latent representations through a pretrained VAE and refinements in pixel space to maximally degrade EHPS predictions while remaining visually imperceptible. By formulating a black-box threat model with API-only access and a strict query budget, UBA injects noise in latent space via $\hat{z} = z + \eta \epsilon_2$ and exploits the Jacobian relation $\Delta x \approx J_{\mathcal{G}_\theta}(z) (\eta \epsilon_2)$, then iteratively optimizes a multi-task loss to balance attack strength and imperceptibility with updates $\Delta x^{(t+1)} = \Delta x^{(t)} + \xi_x \frac{\partial \mathcal{L}}{\partial \mathcal{P}} \frac{\partial \mathcal{P}}{\partial (x + \Delta x^{(t)})} - \lambda (x^{(t)} - x)$. Experiments on 3DPW and UBody show substantial error growth (e.g., average $17.27\%$–$44.49\%$ across models; up to $114.94\%$ MPVPE increase) under black-box conditions, underscoring urgent security considerations for digital human systems. The approach achieves strong attack performance with modest computational resources and a limited query budget, while maintaining perceptual fidelity through latent-space perturbations and a carefully designed multi-task loss; these findings motivate the development of robust defenses for EHPS pipelines.

Abstract

Expressive human pose and shape (EHPS) estimation is vital for digital human generation, particularly in live-streaming applications. However, most existing EHPS models focus primarily on minimizing estimation errors, with limited attention on potential security vulnerabilities. Current adversarial attacks on EHPS models often require white-box access (e.g., model details or gradients) or generate visually conspicuous perturbations, limiting their practicality and ability to expose real-world security threats. To address these limitations, we propose a novel Unnoticeable Black-Box Attack (UBA) against EHPS models. UBA leverages the latent-space representations of natural images to generate an optimal adversarial noise pattern and iteratively refine its attack potency along an optimized direction in digital space. Crucially, this process relies solely on querying the model's output, requiring no internal knowledge of the EHPS architecture, while guiding the noise optimization toward greater stealth and effectiveness. Extensive experiments and visual analyses demonstrate the superiority of UBA. Notably, UBA increases the pose estimation errors of EHPS models by 17.27%-58.21% on average, revealing critical vulnerabilities. These findings underscore the urgent need to address and mitigate security risks associated with digital human generation systems.

Figures (8)

• Figure 1: Overview of the Unnoticeable Black-box Attack (UBA) pipeline. The framework consists of two stages: a) noise injection in latent space, and b) noise enhancement.
• Figure 2: Visualizing various adversarial samples for digital human generation.
• Figure 2: Performance comparison of different adversarial attack methods on state-of-the-art EHPS models on the 3DPW dataset. The error growth rates are marked in gray. The maximum error and maximum error growth rate on each setting are highlightedunderlined.
• Figure 3: Attack performance of different perturbation magnitude $\eta$.
• Figure 4: Attack performance of different query times $t$.