Skull-to-Face: Anatomy-Guided 3D Facial Reconstruction and Editing

TL;DR

This work tackles skull-to-face reconstruction by introducing an anatomy-guided pipeline that combines tissue-depth distribution modeling with diffusion-based initial face generation and a landmark-driven adaptation stage. It defines a Tissue Depth Distribution (TDD) framework, including a global model (TDD-Global) controlled by a single component and a regional model (TDD-Regional) for local refinements, enabling both wide-exploration and precise editing. An anatomy-guided adaptation optimizes a latent face to satisfy facial landmarks while preserving the texture from an initial 2D portrait; the losses integrate landmark, projection, and symmetry terms. On Skull100 and a real skull (Robert the Bruce), the approach achieves accurate geometry, diverse plausible faces, and robust convergence, offering a practical tool for forensic and archaeological applications and enabling interactive exploration of skull-constrained appearances.

Abstract

Deducing the 3D face from a skull is a challenging task in forensic science and archaeology. This paper proposes an end-to-end 3D face reconstruction pipeline and an exploration method that can conveniently create textured, realistic faces that match the given skull. To this end, we propose a tissue-guided face creation and adaptation scheme. With the help of the state-of-the-art text-to-image diffusion model and parametric face model, we first generate an initial reference 3D face, whose biological profile aligns with the given skull. Then, with the help of tissue thickness distribution, we modify these initial faces to match the skull through a latent optimization process. The joint distribution of tissue thickness is learned on a set of skull landmarks using a collection of scanned skull-face pairs. We also develop an efficient face adaptation tool to allow users to interactively adjust tissue thickness either globally or at local regions to explore different plausible faces. Experiments conducted on a real skull-face dataset demonstrated the effectiveness of our proposed pipeline in terms of reconstruction accuracy, diversity, and stability. Our project page is https://xmlyqing00.github.io/skull-to-face-page.

Figures (17)

• Figure 1: Given an input skull scan, based on skull landmarks and tissue depth (blue sticks), facial landmarks (green spheres) can be obtained. The 3D face is inferred under the constraints of facial landmarks.
• Figure 2: An overview of the proposed Skull-to-Face pipeline, which includes three modules. The Face Generation module synthesizes an initial 3D face in consideration of the biological profile of the skull such as age, gender, and ancestry. The Tissue Depth Distribution (TDD) can suggest a valid combination of facial landmarks as geometry constraints. The Anatomy-guided Face adaptation module optimizes the initial face as the facial reconstruction according to the landmark constraints.
• Figure 3: The first principal component can meaningfully provide control on the thickness of tissue depths. (a) shows the tissue depth distribution $C$ in the first principal component. Vertical lines represent three representative values, signifying three different combinations of tissue depths, resulting in three types of face shapes: thin, normal, and fat, as illustrated in (b).
• Figure 4: Examples of the synthesized faces by different attributes.
• Figure 5: TDD-Global model can explore faces with various shapes. From left to right, we iteratively sampled values of distribution $C$, and the corresponding tissue depths increased from short to long. The adapted faces that fit the tissue depth become fatter.