UIKA: Fast Universal Head Avatar from Pose-Free Images

TL;DR

This work introduces a UV-guided avatar modeling strategy, in which each input image is associated with a pixel-wise facial correspondence estimation, and significantly outperforms existing approaches in both monocular and multi-view settings.

Abstract

We present UIKA, a feed-forward animatable Gaussian head model from an arbitrary number of unposed inputs, including a single image, multi-view captures, and smartphone-captured videos. Unlike the traditional avatar method, which requires a studio-level multi-view capture system and reconstructs a human-specific model through a long-time optimization process, we rethink the task through the lenses of model representation, network design, and data preparation. First, we introduce a UV-guided avatar modeling strategy, in which each input image is associated with a pixel-wise facial correspondence estimation. Such correspondence estimation allows us to reproject each valid pixel color from screen space to UV space, which is independent of camera pose and character expression. Furthermore, we design learnable UV tokens on which the attention mechanism can be applied at both the screen and UV levels. The learned UV tokens can be decoded into canonical Gaussian attributes using aggregated UV information from all input views. To train our large avatar model, we additionally prepare a large-scale, identity-rich synthetic training dataset. Our method significantly outperforms existing approaches in both monocular and multi-view settings. Project page: https://zijian-wu.github.io/uika-page/

Figures (14)

• Figure 1: Pipeline Overview. Given a set of unposed input images, our pipeline begins with a facial correspondence estimator that predicts UV coordinates for valid facial pixels, and the corresponding colors are reprojected onto the shared UV space. The source images (screen space) and reprojected images (UV space) are encoded through two dedicated encoders, producing multi-scale features from both screen space and UV space. We then apply screen attention and UV attention to inject these into learnable UV tokens, which are then decoded into UV Gaussian attribute maps while incorporating the aggregated color and confidence map. The resulting canonical Gaussian head avatar supports animation via standard linear blend skinning and achieves real-time rendering at 220 FPS.
• Figure 2: Qualitative results for comparison to baselines in both monocular and multi-view settings in NeRSemble-v2 datasets.
• Figure 3: Qualitative results of different numbers of input views in VFHQ and NeRSemble-v2 dataset.
• Figure 4: Qualitative results for ablation study in the monocular settings in NeRSemble-v2 dataset.
• Figure 5: Qualitative results for in-the-wild cases.