HGC-Avatar: Hierarchical Gaussian Compression for Streamable Dynamic 3D Avatars

TL;DR

The paper addresses the challenge of efficiently encoding and streaming dynamic 3D avatars by leveraging human priors in a hierarchical Gaussian compression framework. It introduces a two-layer architecture: a motion layer using SMPL-X to encode temporal pose and a structural layer using a StyleUNet to map pose maps to Gaussian parameters, with a facial attention mechanism to preserve identity and expression at low bitrates. The authors show that this approach achieves high visual fidelity at substantially lower bitrate than state-of-the-art Gaussian-based methods on THuman4.0, AvatarRex, and ActorsHQ datasets, enabling fast, controllable rendering on edge devices. The work advances practical streaming of immersive 3D avatars and opens avenues for multimodal pose control and interactive holographic applications.

Abstract

Recent advances in 3D Gaussian Splatting (3DGS) have enabled fast, photorealistic rendering of dynamic 3D scenes, showing strong potential in immersive communication. However, in digital human encoding and transmission, the compression methods based on general 3DGS representations are limited by the lack of human priors, resulting in suboptimal bitrate efficiency and reconstruction quality at the decoder side, which hinders their application in streamable 3D avatar systems. We propose HGC-Avatar, a novel Hierarchical Gaussian Compression framework designed for efficient transmission and high-quality rendering of dynamic avatars. Our method disentangles the Gaussian representation into a structural layer, which maps poses to Gaussians via a StyleUNet-based generator, and a motion layer, which leverages the SMPL-X model to represent temporal pose variations compactly and semantically. This hierarchical design supports layer-wise compression, progressive decoding, and controllable rendering from diverse pose inputs such as video sequences or text. Since people are most concerned with facial realism, we incorporate a facial attention mechanism during StyleUNet training to preserve identity and expression details under low-bitrate constraints. Experimental results demonstrate that HGC-Avatar provides a streamable solution for rapid 3D avatar rendering, while significantly outperforming prior methods in both visual quality and compression efficiency.

Figures (7)

• Figure 1: Overview of the proposed framework. Our method consists of three main components: human-prior-based representation, layered compression, and Gaussian-based rendering and interaction. It takes multi-view videos as input, extracts SMPL-X poses, renders pose maps, and uses a facial-oriented StyleUNet to generate Gaussian parameters. At the decoder side, Gaussian reconstruction enables low-latency, controllable avatar rendering driven by video or text inputs.
• Figure 2: User motions are captured by edge devices or provided as motion pose text, enabling pose extraction or generation for novel pose synthesis in the reconstructed character.
• Figure 3: Comparison with reconstruction methods on ActorsHQ dataset icsik2023humanrf. Our method achieves a low bitrate while maintaining excellent reconstruction quality and texture details.
• Figure 4: Comparison with compression methods on THuman4.0 dataset zheng2022structured. Our method maintains the best reconstruction quality even at low bitrates.
• Figure 5: Ablation study on compression quantization step for AvatarReX zheng2023avatarrex and THuman4.0 zheng2022structured shows reconstruction quality degrades with larger steps, with optimal balance achieved at 325KB.