Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Latent Disentanglement in Mesh Variational Autoencoders Improves the Diagnosis of Craniofacial Syndromes and Aids Surgical Planning

Simone Foti, Alexander J. Rickart, Bongjin Koo, Eimear O' Sullivan, Lara S. van de Lande, Athanasios Papaioannou, Roman Khonsari, Danail Stoyanov, N. u. Owase Jeelani, Silvia Schievano, David J. Dunaway, Matthew J. Clarkson

TL;DR

This work opens new avenues to advance diagnosis, aids surgical planning and allows for the objective evaluation of surgical outcomes.

Abstract

The use of deep learning to undertake shape analysis of the complexities of the human head holds great promise. However, there have traditionally been a number of barriers to accurate modelling, especially when operating on both a global and local level. In this work, we will discuss the application of the Swap Disentangled Variational Autoencoder (SD-VAE) with relevance to Crouzon, Apert and Muenke syndromes. Although syndrome classification is performed on the entire mesh, it is also possible, for the first time, to analyse the influence of each region of the head on the syndromic phenotype. By manipulating specific parameters of the generative model, and producing procedure-specific new shapes, it is also possible to simulate the outcome of a range of craniofacial surgical procedures. This opens new avenues to advance diagnosis, aids surgical planning and allows for the objective evaluation of surgical outcomes.

Latent Disentanglement in Mesh Variational Autoencoders Improves the Diagnosis of Craniofacial Syndromes and Aids Surgical Planning

TL;DR

This work opens new avenues to advance diagnosis, aids surgical planning and allows for the objective evaluation of surgical outcomes.

Abstract

The use of deep learning to undertake shape analysis of the complexities of the human head holds great promise. However, there have traditionally been a number of barriers to accurate modelling, especially when operating on both a global and local level. In this work, we will discuss the application of the Swap Disentangled Variational Autoencoder (SD-VAE) with relevance to Crouzon, Apert and Muenke syndromes. Although syndrome classification is performed on the entire mesh, it is also possible, for the first time, to analyse the influence of each region of the head on the syndromic phenotype. By manipulating specific parameters of the generative model, and producing procedure-specific new shapes, it is also possible to simulate the outcome of a range of craniofacial surgical procedures. This opens new avenues to advance diagnosis, aids surgical planning and allows for the objective evaluation of surgical outcomes.
Paper Structure (18 sections, 1 equation, 8 figures)

This paper contains 18 sections, 1 equation, 8 figures.

Table of Contents

  1. Introduction
  2. Methods
  3. Data sources and processing.
  4. Data augmentation with spectral interpolation.
  5. Swap disentangled 3D mesh variational autoencoder.
  6. Manifold visualisation.
  7. Syndrome classification.
  8. Evaluating the effects of different surgical approaches using latent interpolation.
  9. Results
  10. Data augmentation.
  11. Model and latent disentanglement evaluation.
  12. Manifold visualisation.
  13. Syndrome classification.
  14. Evaluating the effects of different surgical approaches.
  15. Discussion
  16. ...and 3 more sections

Figures (8)

  • Figure 1: Mini-batch with swapped attributes. Each shape is showed alongside its latent representation. Both shape attributes and latent subsets are colour-coded to represent different subjects.
  • Figure 2: Manifold visualisation and syndrome classification. Latent vectors obtained encoding shapes with SD-VAE, $\mathbf{z}_P$, can be processed as a whole or in subsets of variables. LDA and QDA models are thus created for each latent subset and for the the whole latent. LDA models are used for dimensionality reduction and manifold visualisation, while QDA models for classification.
  • Figure 3: Example of surgical region definition. In this case, the nose is combined with the upper lip and the nasolabial attributes, thus defining the regions affected by the Le Fort II surgical procedure
  • Figure 4: Evaluation of the data augmentation. Left: rendering of the mesh whose vertices ($\mathbf{X}_\text{aug}$) were obtained with the proposed data augmentation technique from the two real Apert meshes with vertices $\mathbf{X}_1$ and $\mathbf{X}_2$. Centre: silhouettes of the real and augment meshes in the front and side view. The silhouettes are colour-coded like their corresponding meshes. Right: manifold visualisation of SD-VAE trained on real data only.
  • Figure 5: Latent evaluation. Left: visual representation of the $15$ anatomical attributes for which we seek to obtain a disentangled latent representation and the effects of traversing each latent variable (abscissas). The mean distance between a mesh generated with a given variable at its minimum and the one generated with the variable at its maximum is reported for each attribute (ordinates). For each latent variable, we expect a high mean distance in one single attribute and low values for all the others. Right: manifold visualisation of SD-VAE trained on both real and augmented data. The distributions of the latent embeddings are colour-coded according to the different syndromes.
  • ...and 3 more figures