From Pixels to Polygons: A Survey of Deep Learning Approaches for Medical Image-to-Mesh Reconstruction

TL;DR

This survey addresses the challenge of converting medical images into simulation-ready meshes by organizing deep learning approaches into four paradigms: template models, statistical shape models, generative models, and implicit models. It provides a structured taxonomy of methods, loss functions, and evaluation metrics, and conducts a meta-analysis across cardiac and cerebral datasets to compare performance. The study also curates public datasets and discusses practical challenges like topology, multi-modality fusion, and data limitations, offering future directions such as Gaussian splatting and diffusion-based methods. Overall, the work clarifies the landscape of image-to-mesh reconstruction, highlighting the growing role of implicit and generative models in delivering high-fidelity, topology-aware meshes for in-silico trials and personalized medicine.

Abstract

Deep learning-based medical image-to-mesh reconstruction has rapidly evolved, enabling the transformation of medical imaging data into three-dimensional mesh models that are critical in computational medicine and in silico trials for advancing our understanding of disease mechanisms, and diagnostic and therapeutic techniques in modern medicine. This survey systematically categorizes existing approaches into four main categories: template models, statistical models, generative models, and implicit models. Each category is analysed in detail, examining their methodological foundations, strengths, limitations, and applicability to different anatomical structures and imaging modalities. We provide an extensive evaluation of these methods across various anatomical applications, from cardiac imaging to neurological studies, supported by quantitative comparisons using standard metrics. Additionally, we compile and analyze major public datasets available for medical mesh reconstruction tasks and discuss commonly used evaluation metrics and loss functions. The survey identifies current challenges in the field, including requirements for topological correctness, geometric accuracy, and multi-modality integration. Finally, we present promising future research directions in this domain. This systematic review aims to serve as a comprehensive reference for researchers and practitioners in medical image analysis and computational medicine.

Figures (22)

• Figure 1: Pipeline of in-silico trials, consisting of four key stages: image acquisition, reconstruction, generation, and simulation. Medical imaging modalities such as CT and MR scans provide anatomical data, which are then converted into digital twin anatomy in the reconstruction stage. The generation phase creates virtual population anatomy to account for variability, while the simulation stage performs computational analyses to derive model-informed evidence. Two examples are shown: (1) Cerebral aneurysm with flow diverter (sarrami2021silicomacraild2024offlin2023high), where vascular imaging is used to reconstruct the aneurysm, generate a virtual population, and simulate blood flow changes after device implantation; and (2) Aortic Stenosis with transcatheter heart valve (pak2023patientozturk2025ai), where CT images are used to reconstruct the heart, generate a virtual cohort, and simulate hemodynamic effects of THV implantation.
• Figure 2: PRISMA flowchart summarizing the systematic review process. (PRISMA registration number: CRD420250655291)
• Figure 3: Survey structure, categorizing deep learning-based medical surface reconstruction into four paradigms: template models, statistical shape models (SSM), generative models, and implicit models. Dashed boxes represent variables, while solid boxes represent modules that process these variables using algorithms such as convolutional neural networks (CNN) and graph neural networks (GNN). Solid arrows indicate essential processes, whereas dashed arrows denote optional processes.
• Figure 4: Chronological overview of representative template models for medical image-to-mesh reconstruction.
• Figure 5: Schematic of conditioned deformation methods. The framework consists of two pipelines: one leveraging a CNN for deformation feature extraction and the other employing a GNN for direct template mesh deformation. During feature propagation, the CNN pipeline transfers learned features from the image to the GNN pipeline. Dashed boxes represent variables, while solid boxes denote operations. Solid arrows indicate mandatory connections, and dashed arrows represent optional connections.