HyPlaneHead: Rethinking Tri-plane-like Representations in Full-Head Image Synthesis

TL;DR

HyPlaneHead tackles mirroring and seam artifacts in 3D-aware full-head synthesis by introducing a hybrid-plane representation that fuses planar and spherical planes. It combines a unify-split strategy to prevent feature penetration with a near-equal-area, LAEA-based warping (plus elliptical grid mapping) to maximize square feature-map utilization. The method achieves state-of-the-art results on full-head synthesis with reduced artifacts and robust multi-view rendering, supported by comprehensive ablations and inversion experiments. The approach offers a practical, moderately overhead-intensive improvement with broad compatibility to existing 3D-aware GAN pipelines.

Abstract

Tri-plane-like representations have been widely adopted in 3D-aware GANs for head image synthesis and other 3D object/scene modeling tasks due to their efficiency. However, querying features via Cartesian coordinate projection often leads to feature entanglement, which results in mirroring artifacts. A recent work, SphereHead, attempted to address this issue by introducing spherical tri-planes based on a spherical coordinate system. While it successfully mitigates feature entanglement, SphereHead suffers from uneven mapping between the square feature maps and the spherical planes, leading to inefficient feature map utilization during rendering and difficulties in generating fine image details. Moreover, both tri-plane and spherical tri-plane representations share a subtle yet persistent issue: feature penetration across convolutional channels can cause interference between planes, particularly when one plane dominates the others. These challenges collectively prevent tri-plane-based methods from reaching their full potential. In this paper, we systematically analyze these problems for the first time and propose innovative solutions to address them. Specifically, we introduce a novel hybrid-plane (hy-plane for short) representation that combines the strengths of both planar and spherical planes while avoiding their respective drawbacks. We further enhance the spherical plane by replacing the conventional theta-phi warping with a novel near-equal-area warping strategy, which maximizes the effective utilization of the square feature map. In addition, our generator synthesizes a single-channel unified feature map instead of multiple feature maps in separate channels, thereby effectively eliminating feature penetration. With a series of technical improvements, our hy-plane representation enables our method, HyPlaneHead, to achieve state-of-the-art performance in full-head image synthesis.

Figures (9)

• Figure 1: Figure (a, b, c) respectively illustrate the feature map visualizations and geometric structures of the tri-plane, spherical tri-plane, and our proposed hy-plane representation which integrates both planar and spherical planes. Figure (d) shows a head geometry model for defining the coordinate system. Note that in (a, b), the dominant planes ($P_{YZ}$ for tri-plane and $P_{\theta\phi}$ for spherical tri-plane) cause significant inter-channel feature penetration into the other two planes (best viewed when zoomed in), thereby limiting the model's expressiveness. In contrast, (c) resolves this issue entirely by employing a unify-split strategy, where all feature maps are generated within a single channel. As a result, each plane effectively learns its corresponding information without interference from other planes.
• Figure 2: In the tri-plane representation, (a) the feature entanglement issue results in mirroring artifacts, where (b) the back of the head incorrectly exhibits front-face attributes, or (c) the hair's texture and shape display an unnaturally high degree of left-right symmetry. In the spherical tri-plane representation, (d, e) the non-equal-area warping caused by mapping a square to a sphere using $(\theta, \phi)$ coordinates introduces (f) artifacts in the seam and polar regions, as well as uneven spatial feature distribution after warping. By contrast, (g) the tri-plane representation exhibits even spatial feature distribution, whereas (h) the spherical tri-plane representation shows uneven distribution, with features being overly dense in the polar regions and sparse near the equator.
• Figure 3: (a) The Lambert azimuthal equal-area projection (LAEA) opens the South Pole, and maps the sphere to (b) a flat circular plane, with the North Pole at its center. (c) Elliptical grid mapping transforms the circular plane into a square. Conversely, the square can be inversely mapped back to a sphere with near-equal-area properties. (d) In the hy-plane (2+2) representation, two spheres coincide, with their North Poles facing in opposite directions. Please refer to the supplementary videos for a more comprehensive understanding.
• Figure 4: Unify-Split Strategy. (a) Hy-plane (3+1) with evenly splitting; (b) Hy-plane (2+2) with evenly splitting; (c) Hy-plane (3+1) with area-biased splitting; (d) Hy-plane (2+2) with area-biased splitting.
• Figure 5: Qualitative comparison with state-of-the-art methods. (a) Tri-plane representation from chan2022efficient. (b) Tri-grid representation from an2023panohead. (c) Single spherical tri-plane representation from li2024spherehead, where the white dashed box highlights a discontinuity in the hair region. (d, e) Dual spherical tri-plane representation from li2024spherehead. (f–j) Our proposed Hy-plane representation. A closer inspection is recommended for finer details, and we refer readers to the supplementary material for higher-resolution visualizations.