Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An End-to-End Deep Learning Generative Framework for Refinable Shape Matching and Generation

Soodeh Kalaie, Andy Bulpitt, Alejandro F. Frangi, Ali Gooya

TL;DR

A novel unsupervised geometric deep-learning model is developed to establish refinable shape correspondences in a latent space, construct a population-derived atlas and generate realistic synthetic shapes.

Abstract

Generative modelling for shapes is a prerequisite for In-Silico Clinical Trials (ISCTs), which aim to cost-effectively validate medical device interventions using synthetic anatomical shapes, often represented as 3D surface meshes. However, constructing AI models to generate shapes closely resembling the real mesh samples is challenging due to variable vertex counts, connectivities, and the lack of dense vertex-wise correspondences across the training data. Employing graph representations for meshes, we develop a novel unsupervised geometric deep-learning model to establish refinable shape correspondences in a latent space, construct a population-derived atlas and generate realistic synthetic shapes. We additionally extend our proposed base model to a joint shape generative-clustering multi-atlas framework to incorporate further variability and preserve more details in the generated shapes. Experimental results using liver and left-ventricular models demonstrate the approach's applicability to computational medicine, highlighting its suitability for ISCTs through a comparative analysis.

An End-to-End Deep Learning Generative Framework for Refinable Shape Matching and Generation

TL;DR

A novel unsupervised geometric deep-learning model is developed to establish refinable shape correspondences in a latent space, construct a population-derived atlas and generate realistic synthetic shapes.

Abstract

Generative modelling for shapes is a prerequisite for In-Silico Clinical Trials (ISCTs), which aim to cost-effectively validate medical device interventions using synthetic anatomical shapes, often represented as 3D surface meshes. However, constructing AI models to generate shapes closely resembling the real mesh samples is challenging due to variable vertex counts, connectivities, and the lack of dense vertex-wise correspondences across the training data. Employing graph representations for meshes, we develop a novel unsupervised geometric deep-learning model to establish refinable shape correspondences in a latent space, construct a population-derived atlas and generate realistic synthetic shapes. We additionally extend our proposed base model to a joint shape generative-clustering multi-atlas framework to incorporate further variability and preserve more details in the generated shapes. Experimental results using liver and left-ventricular models demonstrate the approach's applicability to computational medicine, highlighting its suitability for ISCTs through a comparative analysis.
Paper Structure (18 sections, 13 equations, 4 figures, 4 tables)

This paper contains 18 sections, 13 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Generative Model: Atlas-R-ASMG
  5. Refinable Attention-based Shape Matching (R-ASM)
  6. Generation Network
  7. Unsupervised Loss
  8. Atlas construction
  9. Joint-Clustering Generative Model: mAtlas-R-ASMG
  10. Clustering Scheme
  11. Generative Modeling
  12. Datasets
  13. Results and Discussion
  14. Evaluation of Atlas-R-ASMG model
  15. Evaluation of Shape Matching
  16. ...and 3 more sections

Figures (4)

  • Figure 1: Diagram illustrating the overview of the Atlas Refinable Attention-based Shape Matching and Generation network (Atlas-R-ASMG) framework, designed for shape generation from shapes with variable mesh topology. Pre-processing involves extracting input shapes of different sizes from 3D images.
  • Figure 2: Scatter plots showing $HD$ (left) and $CD$ (right) results of attention-based shape matching frameworks in $[mm]$, with (a) sGCN-ATT and (b) hGCN-ATT settings, on the $200$ LV and $28$ liver cases test set, comparing the ASM ($x-$axis) versus refinable model R-ASM ($y-$axis). The green/red gradients indicate an increase/decrease in performance with refinement, respectively. Refinement generally improves $HD$ and $CD$ errors of attention-based shape-matching results. Degradation is observed for some LV/liver cases in terms of $HD$, whereas improvements in $HD$ can be significant.
  • Figure 3: Generalisation and Specification ability: Boxplots show the generalisation and specificity errors with $HD$ (in $[mm]$) for the different models RSMP, ASM, and Atlas-R-ASMG with different settings s/h, where "s" and "h" refer to sGCN-ATT-VAE and hGCN-ATT-VAE respectively and the models' performance are statistically compared. The top and bottom rows illustrate results on LV and liver datasets, respectively.
  • Figure 4: Examples of virtual (LV/liver) samples generated by the Atlas-R-ASMG(h) generator model.