Deformation-Recovery Diffusion Model (DRDM): Instance Deformation for Image Manipulation and Synthesis

TL;DR

DRDM introduces a deformation-centric diffusion framework for medical image manipulation and synthesis, replacing intensity-based generation with instance-specific, topology-preserving deformations. It explicitly models deformation fields through forward random deformation diffusion and reverse deformation recovery, enabling diverse yet anatomically plausible transformations suitable for data augmentation and synthetic training. Empirical results on cardiac MRI and pulmonary CT show that DRDM enhances downstream tasks such as few-shot segmentation and image registration, outperforming traditional augmentation baselines and achieving performance close to real-data training. The work demonstrates that deformation diffusion can provide interpretable, clinically relevant generative capabilities with strong potential for broader medical imaging applications.

Abstract

In medical imaging, the diffusion models have shown great potential for synthetic image generation tasks. However, these approaches often lack the interpretable connections between the generated and real images and can create anatomically implausible structures or illusions. To address these limitations, we propose the Deformation-Recovery Diffusion Model (DRDM), a novel diffusion-based generative model that emphasises morphological transformation through deformation fields rather than direct image synthesis. DRDM introduces a topology-preserving deformation field generation strategy, which randomly samples and integrates multi-scale Deformation Velocity Fields (DVFs). DRDM is trained to learn to recover unrealistic deformation components, thus restoring randomly deformed images to a realistic distribution. This formulation enables the generation of diverse yet anatomically plausible deformations that preserve structural integrity, thereby improving data augmentation and synthesis for downstream tasks such as few-shot learning and image registration. Experiments on cardiac Magnetic Resonance Imaging and pulmonary Computed Tomography show that DRDM is capable of creating diverse, large-scale deformations, while maintaining anatomical plausibility of deformation fields. Additional evaluations on 2D image segmentation and 3D image registration tasks indicate notable performance gains, underscoring DRDM's potential to enhance both image manipulation and generative modelling in medical imaging applications. Project page: https://jianqingzheng.github.io/def_diff_rec/

Figures (15)

• Figure 1: (a) Intensity-based diffusion models can synthesize visually realistic images, but lack an explicit relationship with existing real subjects and thus unknown label relationship; (b) In contrast, the proposed Deformation-Recovery diffusion model (DRDM) applies generated deformation fields to real images, representing anatomical variations.These deformations can also be propagated to pixel-wise labels, thus enhancing the utility of the generated data for downstream tasks.
• Figure 2: Overview of the DRDM framework. The model comprises two stages: (i) a deformation diffusion process that applies random deformations within the deformation space, and (ii) a deformation recovery process that recursively estimates and refines deformation field to generate a anatomically realistic image within the learned deformation manifold.
• Figure 3: Illustration of the principle underlying multi-scale random DVF generation and integration in the deformation diffusion process, as detailed in Equation \ref{['eq:phi_enc']} and \ref{['eq:multi_scale_dvf_syn']}. (edited according to Comment-2.3)
• Figure 4: (a) The DRDM is trained at each time step using distance- and angle-based loss function, as described in Algorithm \ref{['algo:train_drdm']}. (b) During inference, deformation fields are generated by the DRDM with varying time step and integrated to produce the final deformation $\phi$ according to Algorithm \ref{['algo:infer_drdm']}. (edited according to Comment-2.3)
• Figure 5: Image and deformation synthesis using the proposed DRDM for few-shot-learning in image segmentation and image registration. (a) Diverse deformation fields, images, and corresponding labels are generated based on the input few images with labels, as described in Algorithm \ref{['algo:data_aug']} and Algorithm \ref{['algo:data_syn']}; (b) The synthesized images and the corresponding labels are used to train a segmentation model, while the generated images and the corresponding DDFs are employed to train a registration model.