Expert Validation of Synthetic Cervical Spine Radiographs Generated with a Denoising Diffusion Probabilistic Model

TL;DR

This work addresses data scarcity in neurosurgical imaging by training a denoising diffusion probabilistic model on an open cervical spine radiograph atlas to produce synthetic lateral cervical spine radiographs. A blinded clinical Turing test with neuroradiologists and spine surgeons shows that synthetic images are not reliably distinguishable from real ones, with 29% identification accuracy and low inter-rater agreement. A memorization audit finds no explicit duplication of real cases, and 20,063 synthetic radiographs are released to support ML tasks such as landmarking, segmentation, and classification. The study suggests that DDPM-generated neuroimaging data can enable privacy-preserving data sharing and large-scale dataset curation, while noting limitations related to resolution, pathology diversity, and automated quality control.

Abstract

Machine learning in neurosurgery is limited by challenges in assembling large, high-quality imaging datasets. Synthetic data offers a scalable, privacy-preserving solution. We evaluated the feasibility of generating realistic lateral cervical spine radiographs using a denoising diffusion probabilistic model (DDPM) trained on 4,963 images from the Cervical Spine X-ray Atlas. Model performance was monitored via training/validation loss and Frechet inception distance, and synthetic image quality was assessed in a blinded "clinical Turing test" with six neuroradiologists and two spine-fellowship trained neurosurgeons. Experts reviewed 50 quartets containing one real and three synthetic images, identifying the real image and rating realism on a 4-point Likert scale. Experts correctly identified the real image in 29% of trials (Fleiss' kappa=0.061). Mean realism scores were comparable between real (3.323) and synthetic images (3.228, 3.258, and 3.320; p=0.383, 0.471, 1.000). Nearest-neighbor analysis found no evidence of memorization. We also provide a dataset of 20,063 synthetic radiographs. These results demonstrate that DDPM-generated cervical spine X-rays are statistically indistinguishable in realism and quality from real clinical images, offering a novel approach to creating large-scale neuroimaging datasets for ML applications in landmarking, segmentation, and classification.

Figures (4)

• Figure 1: Training and Validation Loss of the Denoising Diffusion Probabilistic Model (DDPM). A separate instance of the DDPM was trained from a random sample comprising 85% of the available training data, and the remaining 15% was used for validation. The model loss was computed for the training and validation data partitions at each model checkpoint (every 2,000 training steps).
• Figure 2: Fréchet Inception Distance (FID) Across Training Checkpoints. FID was computed between the 4,963 real clinical images and a set of 7,250 synthetic images generated at every 10th model checkpoint (once every 20,000 training steps).
• Figure 3: Sample Quartet of Real and Synthetic Lateral Cervical Spine X-Ray Images. One of 50 quartets presented to neuroradiologists and spine fellowship-trained neurosurgeons for evaluation. The figure includes one real patient’s X-ray (bottom left) and three synthetic images generated from saved model weight checkpoints: 34 (top left), 40 (bottom right), 80 (top right), where each checkpoint corresponds to 2,000 training steps.
• Figure 4: Comparison of Image Realism Ratings Across Real and Synthetic Datasets. Violin plots display the distribution of ratings for the real data and three synthetic datasets generated from saved model weight checkpoints: 34, 40, 80, where each checkpoint corresponds to 2,000 training steps. p-values are displayed with Holm correction (unadjusted p = 0.128, 0.236, 1.000).