Facial Surgery Preview Based on the Orthognathic Treatment Prediction

TL;DR

This work addresses the challenge of visualizing orthognathic surgery outcomes without extra medical imaging by proposing a fully automated 3D prediction pipeline. It leverages a FLAME-based encoder–decoder, a prediction network for post-change, and four medically inspired losses—mouth-convexity, asymmetry, latent-code, and geometry—along with a data augmentation strategy to handle limited data. Quantitative metrics ($HD$ and $CD$) show the method outperforms prior approaches, and a blinded user study indicates predictions are perceptually indistinguishable from real postoperative results for both clinicians and engineers. The approach yields a practical, patient-oriented visualization tool that can enhance consultations and informed decision-making while avoiding radiation exposure and heavy clinical data requirements.

Abstract

Orthognathic surgery consultation is essential to help patients understand the changes to their facial appearance after surgery. However, current visualization methods are often inefficient and inaccurate due to limited pre- and post-treatment data and the complexity of the treatment. To overcome these challenges, this study aims to develop a fully automated pipeline that generates accurate and efficient 3D previews of postsurgical facial appearances for patients with orthognathic treatment without requiring additional medical images. The study introduces novel aesthetic losses, such as mouth-convexity and asymmetry losses, to improve the accuracy of facial surgery prediction. Additionally, it proposes a specialized parametric model for 3D reconstruction of the patient, medical-related losses to guide latent code prediction network optimization, and a data augmentation scheme to address insufficient data. The study additionally employs FLAME, a parametric model, to enhance the quality of facial appearance previews by extracting facial latent codes and establishing dense correspondences between pre- and post-surgery geometries. Quantitative comparisons showed the algorithm's effectiveness, and qualitative results highlighted accurate facial contour and detail predictions. A user study confirmed that doctors and the public could not distinguish between machine learning predictions and actual postoperative results. This study aims to offer a practical, effective solution for orthognathic surgery consultations, benefiting doctors and patients.

Figures (6)

• Figure 1: Net architecture for predicting the postoperative appearance from a captured 3D scan. During the training phase, the captured mesh and auto-annotated landmarks are first passed through the FLAME fitting procedure, where they are transformed into a compressed latent code. It then passes through a predictor and has the code difference added to it. The FLAME reconstructing procedure helps to calculate well-designed losses using markers and mesh. With the help of medical, latent code, and geological types of loss, the parameter of the code difference predictor can be continuously updated. During the testing phase, the data do not flow through the dashed line. The predicted postoperative appearance is generated through reconstruction using FLAME.
• Figure 2: Side view (left) and frontal view (right) of a orthognathic patient for calculating mouth-convexity loss and asymmetry loss respectively.
• Figure 3: An illustration of synthetic data generation process. The process involves three stages: face segmentation, conditionally generating a synthetic face $F_{gen}$, and stitching the upper and lower regions to create plausible pre- and post-surgery pairs for training.
• Figure 4: Comparison of our method with LARS across four patient cases. On the left, the ground truth (GT) pre- and post-surgical scans are shown for reference. The middle columns display our predicted results, and the right columns show the LARS model's predictions. To compare the prediction errors with the real outcomes, heatmaps are provided showing the error distribution across facial regions, with the error bars located in the bottom-right corner.
• Figure 5: Impact of losses and data augmentation on predictions. The left section illustrates actual surgical data, the middle section presents results after removing specific modules, and the right section showcases the full-model predictions.